JPH1023331A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make joints inconspicuous by devising the effective filtering even when smoothing processing is applied to overlapped joints. SOLUTION: Each of filters 100, 200 is made up of a line memory and an adder. Each of the filters 100, 200 adds signals of horizontal lines adjacent in the vertical direction so as to add the signal of a corresponding horizontal line to image data for each horizontal line belonging to one image by an image pickup element 10 and belonging to another image by an image pickup element 20. Then independent filter processing is applied to each image to obtain outputs Aav and Bav . On the other hand, image data from a memory 13 and image data delayed by a line memory 201 are added by an adder 300 to obtain an output C. Then any of the output C, and the outputs Aav and Bav is selected by a selector 400 in response to a switch command from a CPU 500 and the selected signal is outputted as a video signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing for performing image data processing corresponding to an operation of composing one image by combining (joining) a plurality of images with an overlapped portion. The present invention relates to a device, and is effective when applied to obtain a high-definition image by the above-described synthesis based on an image having a relatively small number of pixels.

[0002]

2. Description of the Related Art FIG. 18 is a conceptual diagram showing how a plurality of images to be subjected to a bonding process are generated by a plurality of image sensors. In FIG. 18, the imaging field of view FVA and the imaging element SB through the optical system LA corresponding to the imaging element SA are shown.
Is adjacent to the imaging field of view FVB through the optical system LB with an overlapping portion W. The imaging field of view F
Each image (image data) corresponding to VA and FVB is an example of an image to be subjected to a bonding (joining) process in this type of image processing apparatus.

FIG. 19 is a schematic diagram showing a state of one continuous image obtained by laminating two images generated by the imaging method as shown in FIG. FIG.
Part (A) shows the state of the image obtained by simply pasting the images corresponding to the imaging visual fields FVA and FVB. In the case (A), when the two images are pasted together, the geometric characteristics and the gradation characteristics of each image are aligned, and the two images are directly joined with a linear boundary. As shown in the figure, there is a problem that a boundary line of a joint portion becomes conspicuous due to a difference in a signal (data) representing an image, that is, a video signal level due to a difference in sensitivity between the two imaging elements SA and SB.

The portion (B) of FIG. 19 is used to reduce the problem that the boundary line becomes conspicuous as in the case (A). It is a schematic diagram which shows the image obtained by performing a special process (smoothing process). A method for obtaining an image like the part (B) is described in, for example, JP-A-57-91.
No. 68 is disclosed.

FIG. 20 is a block diagram showing the configuration of a conventional device. The device shown in FIG. 20 has substantially the same configuration as the device disclosed in the above publication. An image sensor 10 corresponding to the image sensor SA in FIG. 18 and an image sensor 20 corresponding to the image sensor SB in FIG. 18 are provided, and an output signal of the image sensor 10 is supplied to an analog signal processor 11 through an analog signal processing unit 11 including an amplification function unit.
The digital image data is supplied to the D converter 12 and converted into digital image data. The digital image data is stored in the first memory 13. Similarly, an output signal of the image sensor 20 is supplied to an A / D converter 22 through an analog signal processing unit 21 including an amplification function unit, and is converted into digital image data.
This digital image data is stored in the second memory 23.

The output image data read from the first and second memories 13 and 23 is supplied to each corresponding multiplier 14
And 24 are multiplied by the corresponding weighting factors (A) and (B), and the image data (C) and (D)
Is supplied to one input terminal of the adder 30 and the other input terminal of the adder 30, where the signals are added and synthesized.

[0007] The image data (C) and (D) are adjacent to each other with an overlapped portion W in the field of view F of FIG.
It is an image of an area corresponding to each of VA and FVB.

The weighting coefficients (A) corresponding to the respective weighting factors (A) from the mixture ratio distribution unit 40 are provided to the multipliers 14 and 24 in order to weight both data relating to the addition of the image data (C) and (D) in the adder 30. And (B) are supplied. Note that the analog signal processing units 11 and 21 and the mixture ratio distribution unit 40 operate under the control of the CPU 50.

The output image data of the adder 30 is the image data (E) corresponding to the image subjected to the bonding process shown in FIG. This image data (E)
Is supplied to the CPU 50 and recognized here.

FIG. 21 is a timing chart showing a timing relationship of signals (data) of respective parts in the apparatus of FIG. In FIG. 17, (A) to (E) correspond to the signals and data (A) to (E) of the respective units described with reference to FIG. The area indicated by F1 in FIG. 21 as an arrow is an image (image data) related to the output of the image sensor 10 in FIG. 21 and corresponds to the imaging field of view FVA in FIG. 18, and is indicated as F2 in an arrow. The area is an image (image data) related to the output of the image sensor 20 in FIG. 20 and corresponds to the image field of view FVB in FIG. Regions F1 and F
2 has an overlapping portion W corresponding to the overlapping portion W in FIGS. 18 and 19B.

As shown in FIG. 21, one scanning line (C) belonging to one image F1 and another scanning line (D) belonging to another image F2 to be connected to this scanning line (C). In the region corresponding to the overlapping portion W, the weight of the video signal representing the one scanning line (C) becomes closer to the center of the one image F1 (ie, the weighting coefficient A). The weighting and adding process by weighting is such that the weight of the video signal representing the other scanning line (D) (that is, the weighting coefficient B) becomes relatively larger as the position becomes closer to the center of the other image F2. 30 is performed, whereby a smoothing process is performed so as to make the boundary of the joining portion inconspicuous. Note that, in the curve (E) representing the output video data, the portion indicated by the broken line in the above-mentioned W area indicates the level before the corresponding weighting coefficient is multiplied.

As shown, when viewed on the time axis, the image F
The start time of t1 is t1, the end time is t3, the start time of the image F2 is t2, the end time is t4, the start time of the overlapping portion W is t2, and the end time is t3. The weighting coefficient (A) keeps 1 in the section from t1 to t2, and gradually decreases from 1 to 0 during the period from the start point t2 to the end point t3 of the overlapping portion W, while the weighting coefficient (B) is 0 to 1 during the period from the beginning t2 to the end t3 of the part W
And keeps 1 in the section from t3 to t4. In general, the larger the width of the overlapping portion W, the less noticeable the boundary of the connecting portion.

[0013]

FIG. 22 is a diagram showing characteristics of the above-described smoothing processing in the apparatus described with reference to FIGS. 20 and 21. Part (A) of FIG. 22 illustrates a state in which an image F1 having a relatively low level of an image signal and an image F2 having a relatively high level are joined. Part (B) shows the transition of the image signal level when the overlapped portions W are subjected to smoothing processing and connected. Part (C) shows the characteristics when noise is superimposed on the image signal. As shown in the part (C), the noise level of the overlapped portion W on which the smoothing processing has been performed is suppressed by this processing when viewed from the other parts. As shown in part (D), noise may be reduced by about 3 dB in the overlapping portion W. As described above, if the noise level is suppressed only in the overlapped portion subjected to the smoothing process, the joining portion (overlapping portion) becomes more conspicuous in the combined image obtained by the joining process, and the corner smoothing process is performed. The effect of the above will be diminished.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and when a plurality of images including noise are combined (joined) to form one image (image signal), overlapping of the images is performed. It is an object of the present invention to provide a device of this type in which a joint portion (overlapping portion) obtained by performing a smoothing process on a portion does not become more conspicuous.

[0015]

In order to solve the above-mentioned problems, one aspect of the present invention is to perform image data processing corresponding to an operation of combining a plurality of images with overlapping portions and combining them. An image processing apparatus, wherein image data belonging to one image and image data belonging to another image to be joined to the one image are included in the image data corresponding to a portion other than the overlapping portion. Perform filtering processing independently for each image data belonging to each image, for the image data corresponding to the overlapping portion, perform a filtering process based on both image data belonging to each of the one and another image,
An image processing apparatus comprising a smoothing means configured to make a boundary of a connecting portion inconspicuous. …… (1)

Another aspect of the present invention is that the overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means is provided in the vertical direction. Regarding the image data belonging to each corresponding one of the plurality of images arranged side by side and the image data belonging to another image to be joined to this one image, regarding the image data corresponding to the portion other than the above-described overlapping portion, The image data of each horizontal line belonging to the one and other images is subjected to the filtering process independently, and the image data of each horizontal line corresponding to the overlapping portion is belonging to the one and other images. (1) The filter according to the above (1), wherein filtering processing is performed for each horizontal line based on both image data. An image processing apparatus. …………………
………………… (2)

Still another aspect of the present invention is that the overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means is provided in the horizontal direction. Regarding the image data belonging to each corresponding one of the plurality of images arranged side by side and the image data belonging to the other image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion Performs independent filtering processing on the image data of each horizontal line belonging to each of the first and other images, and performs image processing on each of the horizontal lines corresponding to the overlapping portion in the first and other images. The above (1) is characterized in that it is configured to perform filtering processing for each horizontal line based on both of the image data belonging thereto. An image processing apparatus. ……………
……………………… (3)

The features of various aspects of the present invention are listed below.

The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means includes a plurality of images among the plurality of images arranged in the vertical direction. Regarding the image data belonging to each corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion belongs to the one and other images. The image data of each horizontal line is subjected to the filtering process independently on a pixel basis, and the image data of each horizontal line corresponding to the above-mentioned overlapping portion is based on both the image data belonging to the one and the other images. The image processing according to (1), wherein the filtering processing is performed for each horizontal line in pixel units. Equipment ... (4)

The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means includes a plurality of images among the plurality of images arranged in the horizontal direction. Regarding the image data belonging to each corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion belongs to the one and other images. The image data of each horizontal line is subjected to the filtering process independently on a pixel basis, and the image data of each horizontal line corresponding to the above-mentioned overlapping portion is based on both the image data belonging to the one and the other images. The image processing according to (1), wherein the filtering processing is performed for each horizontal line in pixel units. Equipment ....................................
… (5)

An image processing apparatus for performing image data processing corresponding to an operation of combining a plurality of images with an overlapped portion and having the overlapped portion, wherein an area corresponding to the overlapped portion includes the one image; The weight of the video signal representing the one image is relatively greater on the side closer to the center of the image, and the weight of the video signal representing the other image is relatively greater on the side closer to the center of the other image. In an image processing apparatus provided with a smoothing means for performing a weighted addition process by using appropriate weighting so as to make the boundary of the connecting portion inconspicuous, the smoothing means performs processing on image data corresponding to a portion other than the overlapping portion. The filtering process is performed independently for each of the image data belonging to the one and other images, and the image data corresponding to the overlapping portion is Serial image processing apparatus ................................., characterized in that those configured to perform first and the filtering process relies on the image data of both of the belonging to another image (6)

The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means is provided for each of the plurality of images arranged in the vertical direction. Regarding the image data belonging to the corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the horizontal data belonging to the one and other images. The filtering process is performed independently on the image data for each line, and for the image data for each horizontal line corresponding to the overlapping portion, for each horizontal line based on both image data belonging to the one and other images. The image processing apparatus according to the above (6), characterized in that the filtering processing is performed. ................................. (7)

The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means is provided for each of the plurality of images arranged in the horizontal direction. Regarding the image data belonging to the corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the horizontal data belonging to the one and other images. The filtering process is performed independently on the image data for each line, and for the image data for each horizontal line corresponding to the overlapping portion, for each horizontal line based on both image data belonging to the one and other images. The image processing apparatus according to the above (6), characterized in that the filtering processing is performed. ................................. (8)

The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction. The smoothing means is provided for each of the plurality of images arranged in the vertical direction. Regarding the image data belonging to the corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the horizontal data belonging to the one and other images. The filtering process is performed independently for each line of the image data for each pixel, and the image data for each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and the other images. (6) characterized in that filtering processing is performed for each horizontal line in pixel units.
The image processing device described in ………………………………
(9)

The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means is provided for each of the plurality of images arranged in the horizontal direction. Regarding the image data belonging to the corresponding one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the horizontal data belonging to the one and other images. The filtering process is performed independently for each line of the image data for each pixel, and the image data for each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and the other images. (6) characterized in that filtering processing is performed for each horizontal line in pixel units.
Image processing apparatus described in (1)
0)

[0026]

FIG. 1 is a block diagram showing an image processing apparatus according to a first embodiment of the present invention. In FIG. 1, an image sensor 10 corresponding to the image sensor SA in FIG. 18 and an image sensor 20 corresponding to the image sensor SB in FIG. 18 are provided, and an output signal of the image sensor 10 is a known analog signal including an amplification function unit. A / D converter 12 through processing unit 11
And is converted into digital image data, and this digital image data is stored in the first memory 13.
Similarly, the output signal of the image sensor 20 is passed through a well-known analog signal processing unit 21 including an amplification function unit to an A / D converter 22.
And is converted into digital image data, and this digital image data is stored in the second memory 23.

In the subsequent stage of the first memory 13, a line memory 101 and an input signal (data) of the line memory 101 are provided.
And a adder 102 for adding and outputting an output signal (data) delayed by the line memory 101 and outputting the added signal. Similarly, a line is provided at the subsequent stage of the second memory 23. A filter 200 that includes a memory 201 and an adder 202 that adds and outputs an input signal (data) of the line memory 201 and an output signal (data) delayed by the line memory 201 is connected. .

The filters 100 and 200 add signals (data) of horizontal lines adjacent in the vertical direction to belong to one image (F1) by the image sensor 10 and another image (F2) by the image sensor 20. The signal (data) of the corresponding adjacent horizontal line is added to the image data of each horizontal line, and the above-mentioned one image and another image are separated, and an independent filtering process is performed for each image. Output (output of adder 102)
(Aav), the output of the filter 200 (adder 202
(Bav) is obtained as the output of

Image data (An) corresponding to the output of the image pickup device 10 read from the first memory 13;
Image data (Bn) corresponding to the output of the image sensor 20 read from the second memory 23 is supplied to the corresponding filters 100 and 200, respectively. The image data (An) corresponding to the output of the image sensor 10 read from the first memory 13 and the output signal (data) obtained by delaying the image data (Bn) by the line memory 201 are added. The output (C) is obtained by the addition by the detector 300.

The output (Aav) of the filter 100, the output (Bav) of the filter 200, and the output (C) of the adder 300 are selectively switched and output by a selector 400 in response to an output switching command from the CPU 500. It is configured to: The CPU 500 includes an output switching command section 501 for issuing the output switching command.

FIG. 2 is a diagram used for explaining the filtering process in the apparatus of FIG. In the embodiment described with reference to FIG. 2, as shown in FIG. 2B, a plurality (two in this case) of images (F1, F2) have an overlapping portion (W). In this way, they are arranged side by side in the vertical direction and connected. In part (A) of FIG. 2, each line A1, A2, A3, A4, A5, A
6, A7 represent data An for each line, which is an output of the memory 13, and each line B3, B4,
B5, B6, B7, B8, and B9 represent line-by-line data Bn output from the above-described memory 23, and each of the central lines A1 + A2, A2 + A3, B3 + A4, B4 + A5, B5
5 + A6, B5 + A6, B6 + A7, B7 + B8, B8
+ B9 represents data selectively switched and output by the selector 400. As described above, as shown in FIG. 2B, the two images are arranged side by side in the vertical direction.

Part (C) of FIG. 2 corresponds to the device of FIG.
A state in which the filtering process shown in the part (A) of FIG. 2 is performed to obtain a smoothing effect that makes the boundary between the joint portions of the two images inconspicuous can be obtained. It is a figure shown in comparison with the case where it cannot be obtained.

As shown in FIG. 2A and FIG. 2B, the present apparatus employs one of a plurality of (two in the illustrated example) images arranged in the vertical direction. The image data (An) belonging to the image (F1 in the illustrated example) and another image (F1) to be joined to this one image (F1)
Regarding the image data (Bn) belonging to 2), for the image data corresponding to the portion other than the overlapping portion (W), the image data (An, An) for each horizontal line belonging to the one and other images (F1, F2). Bn) independently (F1
(A1 + A2, every F2)
A2 + A3; B7 + B8, B8 + B9), with respect to the image data of each horizontal line corresponding to the overlapping portion, both image data (An, B) belonging to the one and other images, respectively.
n) to perform a filtering process for each horizontal line (B3 + A4, B4 + A5, B5 + A6, B5 + A)
6, B6 + A7).

As a result, as shown in part (C) of FIG. 2, the filtering processing shown in part (A) shown in the left part of FIG. 2 is not performed and a sufficient smoothing effect cannot be obtained. Then, as shown on the right side of the figure, by performing this filtering process, a smoothing effect of making the boundary of the joint portion of the two images inconspicuous is obtained,
The boundary between the joint portions of the two images becomes inconspicuous.

FIG. 3 is a diagram used for describing a filtering process in another embodiment which has the same configuration on the block diagram and a different function from the embodiment described with reference to FIGS. In this embodiment, as shown in FIG. 3B, a plurality of (in this case, two) images (F1,
F2) are arranged side by side in the horizontal direction so as to have an overlapping portion (W) and are connected.

In FIG. 3A, on the left side, an image F1 which is data An for each line which is an output of the memory 13 is shown.
, Lines A1, A2, A3, A4, A5, and A6 are filtered by a filter 100 between vertically adjacent lines, and A1 + A2, A2 + A
The manner in which outputs of horizontal lines 3, 3, A3 + A4, A4 + A5, A5 + A6 are obtained is shown. Also, on the right side of FIG.
n lines B1, B2, B3, B4, B5, B6
Are filtered by the filter 200 between vertically adjacent lines to obtain B1 + B2, B2
The output of a horizontal line of + B3, B3 + B4, B4 + B5, B5 + B6 is shown. (A) Each line A1 + B2, A2 + B3, A3 + B4 in the center of FIG.
A4 + B5 and A5 + B6 represent data selectively switched by the selector 400 and output as the data (C) in FIG. 1 corresponding to the overlapping portion W of both images.

As shown in FIGS. 3A and 3B, in this embodiment, a plurality of images (two in the illustrated example) are horizontally arranged so as to have an overlapping portion (W). The image data (An) belonging to one corresponding image (F1 in the illustrated case) among the plurality of images arranged in the horizontal direction is connected to the one image (F1). With respect to the image data (Bn) belonging to another image (F2), the image data corresponding to the portion other than the overlapping portion (W) is divided into the first and other images (F2).
, F2), the image data (An
, Bn) independently (for each F1; separately for each F2)
Perform filtering processing (A1 + A2, A2 + A3,
A3 + A4, A4 + A5, A5 + A6; B1 + B2, B
2 + B3, B3 + B4, B4 + B5, B5 + B6), and for each horizontal line corresponding to the overlapping portion (W), filtering processing for each horizontal line based on both image data belonging to the one and other images. (A1 + B2, A2 + B3, A3 + B4, A4 + B5
A5 + B6).

The part (C) of FIG. 3 performs the filtering process shown in the part (A) of FIG. 3 in this embodiment to make the boundary between the joint portions of the two images inconspicuous. It is a figure which shows a mode that a smoothing effect is obtained by comparison with the case where such a process is not performed and a smoothing effect is not obtained. As shown in the part (C) of FIG. 3, as compared with the case where the smoothing effect is not obtained without performing the filtering processing shown in the part (A) of FIG. As shown on the upper side, by performing this filtering process, a smoothing effect is obtained that makes the boundary of the joint portion of the two images inconspicuous,
The boundary between the joint portions of the two images becomes inconspicuous.

FIG. 4 is a block diagram showing still another embodiment of the present invention. The embodiment of FIG. 4 is substantially similar to that of FIG. 1 on the block diagram, and is different in that filters 110 and 210 are applied instead of filters 100 and 200 in FIG. . The filter 110 includes a flip-flop 111 for obtaining a delay corresponding to one pixel with respect to an input image signal (data), and an adder 112 for adding the input signal (data) and the output signal (data) of the flip-flop 111 and outputting the added signal. Similarly, the filter 210 outputs a flip-flop 211 that obtains a delay corresponding to one pixel with respect to the input image signal (data), adds the input signal (data) and the output signal (data) of the flip-flop 211 and outputs the result. An adder 212 is included. In the embodiment of FIG. 4, the other parts are the same as those in FIG. 1, and corresponding parts are denoted by the same reference numerals, and description of those parts is omitted.

The filters 110 and 210 add pixel signals (data) adjacent to each other in the horizontal direction, so that horizontal lines belonging to one image (F1) by the image sensor 10 and another image (F2) by the image sensor 20 are added. The data of the corresponding adjacent pixel is added to each image data, and the filtering process is independently performed for each of the above-mentioned one and other images in pixel units, and the output of the filter 110 (the output of the adder 112) is Aav), filter 21
(Bav) is obtained as an output of 0 (output of the adder 212). The image data (An) corresponding to the output of the image sensor 10 read from the first memory 13 and the image data (Bn) corresponding to the output of the image sensor 20 read from the second memory 23 are: , And an adder 300 is added to obtain an output (C).
This is the same as the embodiment.

FIG. 5 is a diagram used for explaining the filtering process in the apparatus of FIG. The embodiment described with reference to FIG. 5 has a plurality of (in this case, two) images (F1, F2) having an overlapping portion (W) as shown in FIG. 5B. In this way, they are arranged side by side in the vertical direction and connected. In part (A) of FIG. 5, each line A11, A12, A13, A14 on the left side in the figure; A21, A
22, A23, A24; ............ A61, A6
2, A63 and A64 represent line-by-line data An (the upper image (F1) at the time of splicing, as shown in the portion (B) of the figure), which is the output of the memory 13, Each line B41, B42, B43, B44; B51, B52,
B53, B54; ............ B91, B92, B
93 and B94 are data B for each line which is an output of the memory 23.
n (as shown in the part (B) of the figure, the lower image (F2) when joining).

Each line A11 + A at the center of the portion (A) in FIG.
12, A12 + A13, A13 + A14, A14 + A15;
... A31 + A32, A32 + A33, A33 + A34,
A34 + A35; ............ A41 + B42, A
42 + B43, A43 + B44, A44 + B45; ...............
A61 + B62, A62 + B63, A63 + B64, A64 +
B65; B71 + B72, B72 + B73, B73 + B74, B74 + B
75; .................. B91 + B92, B92 + B9
3, B93 + B94 and B94 + B95 represent data selectively switched and output by the selector 400.

Of the above, A41 + B42, A42 + B in the middle stage
43, A43 + B44, A44 + B45; ............
…; A61 + B62, A62 + B63, A63 + B64, A64 + B65
Is image data corresponding to the overlapping portion W of the two images.

As shown in FIG. 5A and FIG. 5B, the present apparatus employs one of a plurality of (two in the illustrated example) images arranged in the vertical direction. Of the image data (An) belonging to the image (F1 in the case shown) and the image data (Bn) belonging to another image (F2 in the case shown) joined to this one image (F1). As for the image data corresponding to the portion other than (W), the image data (An, Bn) for each horizontal line belonging to the first and other images (F1, F2) is independently set for each pixel (for each F1; Filtering is performed (separately for each F2) (A11 + A12, A12 + A13, A13 +
A14, A14 + A15; ............ A31 + A
32, A32 + A33, A33 + A34, A34 + A35 ;: B71 + B
72, B72 + B73, B73 + B74, B74 + B75;
B91 + B92, B92 + B93, B93 + B94,
B94 + B95), and filtering processing for each horizontal line based on both image data (An, Bn) belonging to the one and other images is performed on the image data for each horizontal line corresponding to the overlapping portion (A41 + B42). , A42 + B
43, A43 + B44, A44 + B45; ............
…; A61 + B62, A62 + B63, A63 + B64, A64 + B6
5) It is configured as follows.

As a result, as shown in part (C) of FIG. 5, the filtering processing shown in part (A) on the left side of FIG. Then, as shown on the right side of the figure, by performing this filtering process, a smoothing effect of making the boundary of the joint portion of the two images inconspicuous is obtained,
The boundary between the joint portions of the two images becomes inconspicuous.

FIGS. 6 and 7 are the same as those in the embodiment described with reference to FIGS. 4 and 5, but have the same configuration on the block diagram, and are used for describing filtering processing in another embodiment having a different function. It is. In this embodiment, as shown in FIG. 7A, a plurality of (in this case, two) images (F1, F2) are arranged in a horizontal direction so as to have an overlapping portion (W). And join them together.

In FIG. 6, the left side shows the lines AY1, AY2, AY3, AY4 of the image F1 which is the data An for each line, which is the output of the memory 13, and these are horizontally adjacent by the filter 110. A11 + A12, A12 + A13, A14 + A1 after filtering between pixels
4, A14 + A15; ............ A41 + A4
2, the output of a horizontal line consisting of pixel data of A42 + A43, A44 + A44, and A44 + A45 is shown.
Also, on the right side of the figure, each line BY1, BY2, BY3, which is data Bn for each line, which is an output of the memory 23,
BY4 is shown, which is filtered by a filter 210 between horizontally adjacent pixels to obtain B18 + B19,
B19 + B1A, B1B + B1C;...
The output of a horizontal line composed of pixel data of 48 + B49, B49 + B4A, and B4B + B4C is shown. Each line A15 + B16, A16 + B17, A17 + B18 in the center of the figure;
A45 + B46, A46 + B47, A47 +
B48 represents data selectively switched by the selector 400 and output as corresponding to the overlapping portion W of both images. Further, the lower part of the figure (A) shows one horizontal line AX (AX1, AX2,...) Of the image F1.
.., AX8) and one horizontal line BX (BX1, BX2,...) Of the image F2.
BXC) with the overlapping portion W,
9 illustrates a method of adding each pixel data in a filtering process.

As shown in FIGS. 6 and 7A,
In this embodiment, a plurality of images (2
Are arranged in the horizontal direction so as to have an overlapping portion (W), and image data belonging to each corresponding one of the plurality of images arranged in the horizontal direction (F1 in the illustrated case) (An) and image data (Bn) belonging to another image (F2) to be joined to this one image (F1).
), The image data corresponding to the portion other than the overlapping portion (W) is subjected to the first and other images (F1, F2).
2) Image data (An, Bn) for each horizontal line belonging to
) Is filtered independently for each pixel (for each F1; separately for each F2) (A11 + A12, A12 + A
13, A14 + A14, A14 + A15; ............
...; A11 + A42, A42 + A43, A44 + A44, A44 + A4
5; B18 + B19, B19 + B1A, B1B + B1C;
B48 + B49, B49 + B4A, B4B + B4
C), for the image data for each horizontal line corresponding to the overlapping portion (W), filtering processing is performed for each horizontal line on a pixel-by-pixel basis (A15 + B16) based on both image data belonging to the one and the other images. , A16 + B17, A17 + B
18; ……………………… A45 + B46, A46 + B4
7, A47 + B48).

The part (B) of FIG. 7 performs a filtering process as shown in FIG. 6 in this embodiment to obtain a smoothing effect that makes the boundary between the joint portions of the two images inconspicuous. It is a figure which shows a situation in comparison with the case where such a process is not performed and a smoothing effect is not obtained. As shown in the part (B) of FIG. 7, as shown in the lower part of FIG. 7, the filtering processing shown in FIG. 6 is not performed, and a sufficient smoothing effect cannot be obtained. As described above, by performing this filtering process, a smoothing effect of making the boundary of the joint portion of both images inconspicuous is obtained, and the boundary of the joint portion of both images becomes inconspicuous.

FIG. 8 is a block diagram showing still another embodiment of the present invention. The embodiment of FIG. 8 is partially similar in configuration to those of FIGS. 1 and 4 on the block diagram, and means (60) for weighting the outputs of filters 100 and 200 in the configuration of FIG.
0, 610, 620), and means (600, 630, 640) for weighting each value, which is the sum of the outputs of the memory 13 and the line memory 201, with different values. This will be described in more detail below.

In FIG. 8, an image sensor 10 corresponding to the image sensor SA in FIG. 18 and an image sensor 20 corresponding to the image sensor SB in FIG. 18 are provided, and the output signal of the image sensor 10 includes a well-known function including an amplifying function unit. The digital image data is supplied to an A / D converter 12 through an analog signal processing unit 11 and converted into digital image data, and the digital image data is stored in a first memory 13. Similarly, an output signal of the image sensor 20 is supplied to an A / D converter 22 through a known analog signal processing unit 21 including an amplification function unit, and is converted into digital image data. Is stored in

At the subsequent stage of the first memory 13, the line memory 101 and the input signal (data A
n) and an adder 102 for adding and outputting an output signal (data And) and connected thereto. Similarly, a line memory 201 and the line A filter 200 including an adder 202 for adding and outputting an input signal (data Bn) and an output signal (data Bnd) of the memory 201;
Is connected. The above is the same configuration as in FIG.

The filters 100 and 200 add horizontal line signals that are adjacent in the vertical direction, so that each of the horizontal lines belonging to one image (F1) by the image sensor 10 and the other image (F2) by the image sensor 20 is added. The signal (data) of the corresponding adjacent horizontal line is added to the image data, and an independent filtering process is performed for each of the above-described one and other images. As an output of the filter 100 (an output of the adder 102), (Aav) , Filter 20
(Bav) is obtained as an output of 0 (output of the adder 202).

Image data (An) corresponding to the output of the image sensor 10 read from the first memory 13;
Image data (Bn) corresponding to the output of the image sensor 20 read from the second memory 23 is supplied to the corresponding filters 100 and 200, respectively. The image data (An) corresponding to the output of the image sensor 10 read from the first memory 13 and the data obtained by delaying the image data (Bn) by one horizontal line by the line memory 201 are as follows: The output (C) is obtained by addition by the adder 300.

The outputs (Aav) of the filter 100 and the output (Bav) of the filter 200 are multiplied by weighting coefficients k and m by corresponding multipliers 610 and 620, respectively. Further, the output (C) of the adder 300 is multiplied by the weighting coefficients 〓 and n by the respective multipliers 630 and 640 to obtain different outputs. These weighting coefficients k and m and 〓 and n are generated and supplied by the mixture ratio distributor 600.

The outputs (D and E) of multipliers 610 and 630 are added by adder 310, and the outputs (F and G) of multipliers 620 and 640 are similarly added to adder 31.
0 is added, and the outputs of the adders 310 and 320 are added by the adder 330 to obtain an output (H).

FIG. 9 is a timing chart showing a timing relationship of signals (data) of respective parts in the apparatus of FIG. The horizontal axis is a time axis. In the illustrated embodiment, the horizontal axis is a process corresponding to an operation of joining two images with an overlapping portion W in the horizontal direction. In FIG. 9: An is memory 1
3 is output (one input of the adder 102); And is the output of the line memory 101 (the other input of the adder 102);
av is the output of adder 102 (multiplicand input of multiplier 610); Bn is the output of memory 23 (one input of adder 202); Bnd is the output of line memory 201 (adder 20).
Bav is the output of the adder 202 (the multiplicand input of the multiplier 620); C is the output of the adder 300 (the multiplicand input of the multipliers 630 and 640); k is the multiplier input of the multiplier 610 ( Weighting coefficient output of the mixture ratio distributor 600);
1 is a multiplier input of the multiplier 630 (weighting coefficient output of the mixing ratio distributor 600); m is a multiplier input of the multiplier 620 (weighting coefficient output of the mixing ratio distributor 600); n is a multiplier 64
A multiplier input of 0 (weighting coefficient output of the mixture ratio distributor 600); D is an output of the multiplier 610 (one input of the adder 310); E is an output of the multiplier 630 (other input of the adder 310); F is an output of the multiplier 620 (one input of the adder 320); G is an output of the multiplier 640 (the other input of the adder 320); H is an output of the adder 330 (video signal output of the present apparatus). is there.

As shown in FIG. 9, the mixing ratio distributor 60
The weighting coefficient k from 0 gradually decreases in seven steps from the beginning of the overlapping portion W of the two images F1 and F2 on the F1 side toward the center of the overlapping portion W, and conversely, the weighting factor 1 is It gradually increases in seven steps from the start end on the F1 side toward the center of the overlapping portion W. Further, the weighting coefficient m gradually increases in seven steps from the center of the overlapping portion W toward the end of the overlapping portion W on the F2 side, and conversely, the weighting coefficient n increases from the center of the overlapping portion W to the F2 side of the overlapping portion W. Gradually increases in seven steps toward the end of.

Since the weighting coefficients from the mixing ratio distributor 600 change as described above, the D, E, F, and G resulting from multiplying these coefficients also change according to the change in these coefficients. Present. Therefore, the degree of influence (degree of influence) due to the characteristics of both images F1 and F2 changes smoothly in the overlapping portion W of the connection to the output H resulting from the addition of D, E, F, and G. Smoothing processing is performed so that the joined portions are not noticeable.

FIGS. 10 and 11 are diagrams used for describing the filtering processing in the embodiment described with reference to FIGS. 8 and 9. FIG. This embodiment corresponds to FIG.
As shown in the section, a plurality of (two in this case) images (F
1, F2) are arranged side by side in the horizontal direction so as to have an overlapping portion (W) and are connected. In the part (A) of FIG. 10, each line A1, A shown on the left side
2, A3 and A4 represent data An for each line which is an output of the memory 13 in FIG. 8, and A1 + A2, A2 + A3 and A3
+ A4 is the output data of the adder 102 (that is, the filter 10
(Output of 0) represents the filtered data Aav for each line. In the apparatus of FIG. 8, the final output (H) is obtained as the output of the adder 330, but each weighting coefficient (k,
l, n, m) as described with reference to FIG. 8, as a result, data having a value equal to the output of the filter 100 becomes the left image (F1, F2) of the two images (F1, F2). F1) is output as data representing a portion other than the overlapping portion (W).

Further, B1, B2 and B2 shown on the right side of FIG.
B3 and B4 represent line-by-line data Bn output from the memory 23 in FIG. 7, and B1 + B2, B2 + B3, B3 + B
Reference numeral 4 denotes filtered data Bav for each line as output data of the adder 202 (that is, output of the filter 200). Similarly to the description of the left image (F1), the output of the filter 200 is used as data representing a portion of the two images (F1 and F2) other than the overlapping portion (W) of the right image (F2). Will be output.

Each weighting coefficient (k, l, n, m) from the mixture ratio distributor 600 changes in seven steps as described with reference to FIG.
= 1 and k = m = 0, the filtering output of each line is A1 + B2, A2 + B3, A3 + B4,...
Becomes On the left side of the center WM, as you move to the left,
A1 + A2, A2 + A3, or A3 +
The data as shown in the figure has a large weighting of A4, and the data as shown in the drawing has a larger weighting of B1 + B2, B2 + B3 or B3 + B4 in the successive lines toward the right side from the center WM.

Part (A) of FIG. 11 is a waveform diagram showing a change in S / N due to data processing (smoothing) in the embodiment described with reference to FIGS. In the same figure, the upper part is S when the left and right images are simply connected.
FIG. 10 is a diagram showing a state of change of / N, in which noise is suppressed most at the central portion WM of the overlapping portion W and S / N is improved.
For this reason, there is a problem that the central portion WM of the overlapping portion W stands out as a joining boundary.
On the other hand, the lower part of the figure is the result of performing filtering by the characteristic configuration according to the present embodiment described with reference to FIG. 10A and performing the smoothing. There is no change in S / N at the central portion WM and the left and right ends of the W, and the S / N is slightly improved at each portion between the central portion WM and the left and right ends. Since the area where the S / N ratio is small and thus is divided into two, the boundary of joining becomes less noticeable.

Part (B) of FIG. 11 is a diagram showing a state of a difference in S / N characteristics between the conventional method described in part (A) of FIG. 11 and the method of the embodiment. As shown in the drawing, the S / N is improved by about 3 dB in the central portion WM of the overlapping portion W in the conventional method, but the S / N is improved by 1.25 in the method of the embodiment.
A region where the S / N ratio is improved and the S / N ratio is improved is divided into two portions, that is, a portion between the central portion WM and both right and left ends.

In the embodiment described with reference to FIGS. 10 and 11, from one aspect, adjacent images (F1, F
In the area corresponding to the overlapping portion (W) of 2), the weight of the video signal representing the one image is relatively large toward the center of one image (F1), and the center of the other image (F2) is relatively large. An image provided with a smoothing means for performing a weighting addition process by weighting such that the weight of the video signal representing the other image becomes relatively larger as the side closer to the portion makes the boundary of the connecting portion inconspicuous In the processing device, the image data corresponding to the portion other than the overlapping portion (W) is independently set for each of the image data belonging to the first and other images (F1, F2) (for each F1; separately for each F2). The image processing apparatus performs filtering processing, and performs filtering processing on image data corresponding to the overlapping portion (W) based on both image data belonging to the one and other images. Those were

FIG. 12 is a diagram used for explaining the filtering processing in the apparatus of FIG. In the embodiment described with reference to FIG. 12, a plurality of (two in this case) images (F1, F2) have an overlapping portion (W), as shown in FIG. In this way, they are arranged side by side in the vertical direction and connected. In the part (A) of FIG. 12, each line A1, A2, A3, A
4, A5 and A6 represent data An for each line which is an output of the memory 13 in FIG.
B4, B5, B6, B7, and B8 represent line-by-line data Bn output from the memory 23 of FIG. 8, and each of the central lines A1 + A2, A2 + A3, (A3 + B3) / 2 + A4
B4 + A5, B5 + (A6 + B6) / 2, B6 + A7,
B7 + B8 are output from the mixing ratio distributing unit 600 shown in FIG. 8 from the respective lines multiplied by the weighting coefficients (k, l, n, m) through the addition combining and filtering (smoothing processing) by the adder 330. Represents the data to be processed.

The part (C) of FIG. 12 performs the filtering process as shown in the part (A) of FIG. 12 in the apparatus of FIG. 8 so as to make the boundary between the joint portions of the two images inconspicuous. In the case where the smoothing effect is obtained (right), without performing such filtering processing, the S / N is improved only in the overlapping portion (W) by smoothing, and the effect of making the boundary inconspicuous is reduced (left). It is a figure shown by comparison with.

The smoothing means for performing the smoothing process described with reference to FIG.
Among a plurality of images (F1, F2) arranged in the vertical direction with an overlapping portion (W), image data (An) belonging to each corresponding one image and belonging to another image to be joined to this one image. With respect to the image data (Bn), the image data (An
, Bn) for each horizontal line belonging to the first and other images independently (for each F1; Fn).
2 for each of the horizontal lines corresponding to the overlapping portion (W), and for each of the horizontal lines based on both the image data belonging to the one and the other images. It is configured to perform a filtering process.

FIG. 13 is a block diagram showing still another embodiment of the present invention. Although the configuration on the block diagram is almost the same as the embodiment of FIG. 8 described above, the filter 100 (the line memory 101 and the adder 102) in FIG.
The filter 110 (the flip-flop 111 and the adder 112) described with reference to FIG. 4 is used instead of the filter 200, and the filter 200 (the line memory 201) shown in FIG.
Filter 210 (flip-flop 211 and adder 2) described with reference to FIG.
12) is used.

In FIG. 13, an image sensor 10 corresponding to the image sensor SA in FIG. 18 and an image sensor 20 corresponding to the image sensor SB in FIG. 18 are provided, and an output signal of the image sensor 10 includes a well-known function including an amplification function unit. Analog signal processing unit 11
Is supplied to the A / D converter 12 to be converted into digital image data, and this digital image data is stored in the first memory 13. Similarly, the output signal of the image sensor 20 is converted into a known analog signal processing unit 21 including an amplification function unit.
Is supplied to the A / D converter 22 to be converted into digital image data, and this digital image data is stored in the second memory 23.

The subsequent stage of the first memory 13 includes a flip-flop 111 and an adder 112 for adding and outputting an input signal (data An) and an output signal (data And) of the flip-flop 111. Similarly, a flip-flop 211 and the flip-flop 21 are provided at the subsequent stage of the second memory 23.
1 input signal (data Bn) and output signal (data Bn).
nd), and a filter 210 including an adder 212 for adding and outputting the sum. The above is the same configuration as in FIG.

The filters 110 and 210 add one pixel (F1) of the image sensor 10 and the image data of the image sensor 20 by adding pixel data adjacent in the horizontal direction.
The image data of each horizontal line belonging to the resulting image (F2) is subjected to independent filtering processing (for each F1; for each F2) for each of the above-mentioned one and other images on a pixel-by-pixel basis. (Aav) as the output of the adder 112 (the output of the adder 112),
(Bav) is obtained as (output of 12).

Image data (An) corresponding to the output of the image sensor 10 read from the first memory 13;
Image data (Bn) corresponding to the output of the image sensor 20 read from the second memory 23 is supplied to the corresponding filters 110 and 210, respectively. Further, the image data (An) corresponding to the output of the image sensor 10 read from the first memory 13 and the output data obtained by delaying the image data (Bn) by one pixel by the flip-flop 211 by a time corresponding to one pixel , Adder 300 to obtain an output (C).

The outputs (Aav) of the filter 110 and the output (Bav) of the filter 210 are multiplied by weighting coefficients k and m by the corresponding multipliers 610 and 620, respectively. In addition, each of the multipliers 63 outputs one data which is the output (C) of the adder 300.
The weighting coefficients 得 and n are multiplied by 0 and 640 to obtain different outputs. These weighting coefficients k and m and 〓 and n are generated and supplied by the mixture ratio distributor 600.

The outputs (D and E) of multipliers 610 and 630 are added by adder 310, and the outputs (F and G) of multipliers 620 and 640 are similarly added to adder 31.
0 is added, and the outputs of the adders 310 and 320 are added by the adder 330 to obtain an output (H).

The timing relationship of the signals (data) of each part in the device of FIG. 13 is the same as that of FIG. 9, but when the operation of the device of FIG. Will be shown again. FIG. 9 to FIG.
When it is seen as a representation of the timing relationship between the signals (data) of each part in the device of
(One input of the adder 112); And is the output of the flip-flop 111 (the other input of the adder 112);
Aav is the output of adder 112 (multiplicand input of multiplier 610); Bn is the output of memory 23 (one input of adder 212); Bnd is the output of line memory 201 (adder 21
Bav is the output of the adder 202 (the multiplicand input of the multiplier 620); C is the output of the adder 300 (the multiplicand input of the multipliers 630 and 640); k is the multiplier input of the multiplier 610 ( Weighting coefficient output of the mixture ratio distributor 600);
1 is a multiplier input of the multiplier 630 (weighting coefficient output of the mixing ratio distributor 600); m is a multiplier input of the multiplier 620 (weighting coefficient output of the mixing ratio distributor 600); n is a multiplier 64
A multiplier input of 0 (weighting coefficient output of the mixture ratio distributor 600); D is an output of the multiplier 610 (one input of the adder 310); E is an output of the multiplier 630 (other input of the adder 310); F is an output of the multiplier 620 (one input of the adder 320); G is an output of the multiplier 640 (the other input of the adder 320); H is an output of the adder 330 (video signal output of the present apparatus). is there.

Also in the embodiment shown in FIG. 13, as shown in FIG. 9, the weighting coefficient k from the mixture ratio distributor 600 is such that the overlapping portion W of the two images F1 and F2 overlaps with the overlapping portion W from the starting end on the F1 side. The weighting coefficient 1 gradually increases in seven steps toward the center of the overlapped portion W from the start end of the overlapped portion W on the F1 side toward the center of the overlapped portion W. Also, the weighting coefficient m gradually increases in seven steps from the center of the overlapping portion W to the end of the overlapping portion W on the F2 side,
Conversely, the weighting coefficient n gradually increases in seven steps from the center of the overlapping portion W to the end of the overlapping portion W on the F2 side.

FIGS. 14 and 15 are diagrams used for explaining the filtering process in the embodiment described with reference to FIG. 13 and referring to FIG. In this embodiment, a plurality of (in this case, two) images (F1, F2) are arranged in a horizontal direction so as to have an overlapping portion (W), as shown in FIG. It is something to connect. FIG.
In FIG. 4, on the left side, each line (AY) of the image F1 which is the data An for each line, which is the output of the memory 13 in FIG.
1), (AY2), and (AY3) are shown in FIG.
The filter (AY1) which is subjected to the filtering process between the pixels adjacent in the horizontal direction by the third filter 110 is A
In the figure, the output of horizontal lines A21 + A22 and A22 + A23 for the line of (AY2); A31 + A32 and A32 + A33 for the line of (AY3) is obtained independently for each pixel for the image F1. 13, each line (BY1), which is data Bn for each line, which is an output of the memory 23 in FIG.
(BY2) and (BY3) are shown, which are filtered by the filter 210 of FIG. 11 between pixels adjacent in the horizontal direction, and the (BY1) line is B1A + B1.
B, B1B + B1C; (BY2) line is B2A +
B2B, B2B + B2C; (BY3) line is B3A
The output of the horizontal line + B3B, B3B + B3C is shown.

In FIG. 14, focusing on the WM at the center of the overlapping portion W of both images, the respective weighting coefficients (k, l, n, m) from the mixture ratio distributor 600 are as described with reference to FIG. It changes in seven steps, and in this central part WM, l =
Since n = 1 and k = m = 0, the filtering output of each pixel is A16 + B17; A26 + B27; A36 + B
37, ... In addition, the weighting of pixels shifted to the left and to the left on the left side of the center portion WM is relatively large, and the weighting of pixels shifted to the right on the right side of the center portion WM is relatively large. The data becomes as large as shown in the figure.

Part (B) of FIG. 15 is a waveform diagram showing a change in S / N due to data processing (filtering) in the embodiment described with reference to FIG. 13 using FIG. In the same figure, the upper side is a diagram showing the state of change of S / N when the left and right images are simply joined together. The noise is suppressed most at the central portion WM of the overlapping portion W, and the S / N is improved. I have. For this reason, there is a problem that the central portion WM of the overlapping portion W stands out as a joining boundary. On the other hand, the lower part of the figure is the result of performing filtering by the characteristic configuration according to the present embodiment described with reference to FIG. 14, and as shown, the central part WM and the left and right ends of the overlapping part W are shown. There is no change in S / N in the part,
S / N slightly in each part between the center part WM and the left and right ends
As a result, the degree of improvement in S / N is small, and the region in which S / N is improved is divided into two, so that the boundary of joining becomes less noticeable.

Part (C) of FIG. 15 is a diagram showing the difference in S / N characteristics between the conventional method described in FIG. 15 (B) and the method of the embodiment. As shown in the drawing, the S / N is improved by about 3 dB in the central portion WM of the overlapping portion W in the conventional method, but the S / N is improved by 1.25 in the method of the embodiment.
A region where the S / N ratio is improved and the S / N ratio is improved is divided into two portions, that is, a portion between the central portion WM and both right and left ends.

In the embodiment described with reference to FIGS. 14 and 15, from one aspect, adjacent images (F1, F1
In the area corresponding to the overlapping portion (W) of 2), the weight of the video signal representing the one image is relatively large toward the center of one image (F1), and the center of the other image (F2) is relatively large. An image provided with a smoothing means for performing a weighting addition process by weighting such that the weight of the video signal representing the other image becomes relatively larger as the side closer to the portion makes the boundary of the connecting portion inconspicuous In the processing device, the image data corresponding to the portion other than the overlapping portion (W) is independently set for each of the image data belonging to the first and other images (F1, F2) (for each F1; separately for each F2). A filtering process is performed on a pixel-by-pixel basis, and the image data corresponding to the overlapping portion (W) is filtered on a pixel-by-pixel basis based on both the image data belonging to the one and the other images. It is configured to perform the management.

FIGS. 16 and 17 are diagrams having the same configuration on the block diagram as the device of FIG. 13 and used for describing filtering processing in the device of another embodiment having a different function. In the embodiment described with reference to FIGS. 16 and 17, as shown in FIG. 17A, a plurality (two in this case) of images (F1, F2) are overlapped (W). ) And are arranged side by side in the vertical direction and connected. In FIG. 16, each line A11, A12, A13; A2 on the left side of the figure (when connected, the upper image F1 is constituted as shown in FIG. 17A).
1, A22, A23; ............ A91, A9
2. A93 is data An for each line which is an output of the memory 13.
Each of the lines B31, B32, B33; B41, B42, B43; on the right side (constituting the lower image F2 as shown in FIG.
... BB1, BB2, BB3 represent data Bn for each line which is an output of the memory 23.

Each line at the center of FIG. 16 includes an overlapping portion W for joining, and as shown, A11 + A
12, A12 + A13, A13 + A14; A21 + A22, A22 + A2
3, A23 + A24; {3 (A31 + A32) + A31 + B32}
/ 4, {3 (A32 + A33) + A32 + B33} / 4, $ 3
(A33 + A34) + A34 + B34｝ / 4; (A41 + A42 + A
41 + B42) / 2, (A42 + A43 + A42 + B43) / 2
(A43 + A44 + A43 + B34) / 2; ………………………
A61 + B62, A62 + B63, A63 + B64; $ 3
(A71 + B72) + B71 + B72｝ / 4,...; (A81
+ B82 + B81 + B82) / 2,..., A91 + B92 +
3 (B91 + B92)｝ / 4,…, ……………………
………; BA1 + BA2, BA2 + BA3, BA3 + BA4; BB1
+ BB2, BB2 + BB3, BB3 + BB4. That is, in the configuration of the block diagram as shown in FIG. 13, the respective weighting coefficients k, l, n, and m output from the mixture ratio distributor 600 gradually change in seven steps as shown by k, l, n, and m in FIG. Therefore, the image data H obtained as an output of the adder 330 is like this.

As shown in FIGS. 16 and 17 (A), the present apparatus uses one image (two in the illustrated example) corresponding to one of a plurality of images (two images in the figure) arranged vertically. In the case shown, the image data (An) belonging to F1) and another image (F2) to be joined to this one image (F1)
, The image data (An, Bn) for each horizontal line belonging to the first and other images (F1, F2) for image data corresponding to portions other than the overlapping portion (W). Is configured to perform filtering processing independently for each pixel (for each F1; for each F2).

As a result, as shown in the part (B) of FIG. 17, the filtering process shown in FIG. 16 is not performed as shown on the left side of FIG. As shown on the right side of the figure, a smoothing effect that makes the degree of noise suppression uniform and makes the boundary between the joints of the two images inconspicuous as shown on the right side of FIG. Is obtained.

In each of the embodiments described above with reference to the block diagrams of FIGS. 8 and 13, if the weighting coefficient supplied from the mixture ratio distributor 600 is set to 0, the data corresponding to the weighting coefficient is not selected. Since they are equivalent, a simple configuration in which a selector for selecting data according to an overlapping portion as in the other embodiments and a portion other than this portion is omitted is realized.

The inventions seen in various aspects included in the present application will be listed below, and their effects will be described together.

(1) An image processing apparatus for performing image data processing corresponding to an operation of combining a plurality of images with an overlapped portion and by combining the images, wherein image data belonging to one image and one With respect to image data belonging to another image to be joined to the image, image data corresponding to a portion other than the overlapping portion is subjected to filtering processing independently for each image data belonging to the one and other images, and For image data corresponding to the above-mentioned one and another image, performing a filtering process based on both of the image data belonging to the above-mentioned one and the other image, so as to make the boundary of the joining portion inconspicuous. Characteristic image processing device.

In the technique prior to the invention (1), when a plurality of images including noise are combined (joined) to form one image (image signal), a smoothing process is performed on an overlapping portion of the combination. There has been a problem that the connecting portion (overlapping portion) becomes more conspicuous. According to the invention (1), such a problem can be solved.

(2) The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means includes a plurality of images arranged in the vertical direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. The filtering process is independently performed on the image data of each horizontal line belonging to each, and the image data of each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and other images. The image processing apparatus according to (1), wherein the image processing apparatus is configured to perform a filtering process for each horizontal line.

According to the invention of (2), among the general effects of the invention of (1), in particular, when images are arranged and connected in the vertical direction, the connecting portion (overlapping portion) is inconspicuous. can do.

(3) The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means includes a plurality of images arranged in the horizontal direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. The filtering process is independently performed on the image data of each horizontal line belonging to each, and the image data of each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and other images. The image processing apparatus according to (1), wherein the image processing apparatus is configured to perform a filtering process for each horizontal line.

According to the invention of the above (3), among the general effects of the invention of the above (1), particularly when the images are arranged in the horizontal direction and connected, the connection portion (overlapping portion) is inconspicuous. can do.

(4) The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means includes a plurality of images arranged in the vertical direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. Filtering is performed independently on a pixel-by-pixel basis for each horizontal line of image data, and both image data belonging to the first and other images are processed for image data of each horizontal line corresponding to the overlapping portion. Wherein the filtering process is performed for each horizontal line on a pixel-by-pixel basis. Processing apparatus.

According to the invention of the above (4), among the general effects of the above invention of the above (1), the present invention is particularly suited to the case where images are arranged in the vertical direction and connected, and in addition to the effects of the above invention of (1). In particular, since filtering is performed on a pixel-by-pixel basis, smoother smoothing processing can be performed over the entire image to make joint portions (overlapping portions) less noticeable.

(5) The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means includes a plurality of images arranged in the horizontal direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. Filtering is performed independently on a pixel-by-pixel basis for each horizontal line of image data, and both image data belonging to the first and other images are processed for image data of each horizontal line corresponding to the overlapping portion. Wherein the filtering process is performed for each horizontal line on a pixel-by-pixel basis. Processing apparatus.

According to the invention of the above (5), among the general effects of the above invention of the above (1), the present invention is particularly suitable for the case where images are arranged in the horizontal direction and connected, and in addition to the effects of the above invention of (1). In particular, since filtering is performed on a pixel-by-pixel basis, smoother smoothing processing can be performed over the entire image to make joint portions (overlapping portions) less noticeable.

(6) An image processing apparatus for performing image data processing corresponding to an operation of combining a plurality of images with an overlapped portion and having the overlapped portion. The weight of the video signal representing the one image is relatively large near the center of the image, and the weight of the video signal representing the other image is relatively large near the center of the other image. In an image processing apparatus including a smoothing unit that performs a weighting addition process by weighting so as to make the boundary of the joining unit inconspicuous, the smoothing unit includes:
The image data corresponding to the portion other than the overlapping portion is subjected to the filtering process independently for each of the image data belonging to the first and other images, and the image data corresponding to the overlapping portion is subjected to the first and other images. An image processing apparatus configured to perform a filtering process based on both image data belonging to each of the image data.

In the technique prior to the invention (6), when a plurality of images including noise are combined (joined) to form one image (image signal), a smoothing process is performed on an overlapping portion of the combination. There has been a problem that the connecting portion (overlapping portion) becomes more conspicuous. According to the invention of (6), such a problem can be solved. Since the weighted addition processing is performed at the overlapping part of the bonding, smoother smoothing processing is performed over the entire image.

(7) The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means operates on the plurality of images arranged in the vertical direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. The filtering process is independently performed on the image data of each horizontal line belonging to each, and the image data of each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and other images. The image processing apparatus according to (6), wherein the image processing apparatus is configured to perform a filtering process for each horizontal line.

According to the invention of (7), among the general effects of the invention of (6), particularly when the images are arranged in the vertical direction and connected, the connecting portion (overlapping portion) is inconspicuous. can do.

(8) The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means includes a plurality of images arranged in the horizontal direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. The filtering process is independently performed on the image data of each horizontal line belonging to each, and the image data of each horizontal line corresponding to the overlapping portion is based on both the image data belonging to the one and other images. The image processing apparatus according to (6), wherein the image processing apparatus is configured to perform a filtering process for each horizontal line.

According to the invention of (8), among the general effects of the invention of (6), in particular, when images are arranged in a horizontal direction and connected, the connecting portion (overlapping portion) is inconspicuous. can do.

(9) The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means includes a plurality of images arranged in the vertical direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. Filtering is performed independently on a pixel-by-pixel basis for each horizontal line of image data, and both image data belonging to the first and other images are processed for image data of each horizontal line corresponding to the overlapping portion. (6) wherein the filtering process is performed for each horizontal line on a pixel-by-pixel basis. Processing apparatus.

According to the invention (9), among the general effects according to the invention (6), the present invention is particularly suitable for the case where images are arranged in the vertical direction and connected, and in addition to the effects according to the invention (6). In particular, since filtering is performed on a pixel-by-pixel basis, smoother smoothing processing can be performed over the entire image to make joint portions (overlapping portions) less noticeable.

(10) The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in the horizontal direction, and the smoothing means includes a plurality of images arranged in the horizontal direction. Of the image data belonging to each applicable one image and the image data belonging to another image to be joined to this one image, the image data corresponding to the portion other than the overlapping portion is the same as the one and the other images. Filtering is performed independently on a pixel-by-pixel basis for each horizontal line of image data, and both image data belonging to the first and other images are processed for image data of each horizontal line corresponding to the overlapping portion. (6) wherein the filtering process is performed on a pixel-by-pixel basis for each horizontal line. An image processing device.

According to the invention of (10), (6)
Among the general effects according to the invention of the present invention, the present invention is particularly suitable for the case where images are arranged in a horizontal direction and connected, and in addition to the effect of the above-mentioned invention (6), in particular, since filtering is performed on a pixel-by-pixel basis, the whole image is further smoothed. By performing a smoothing process, a joining portion (overlapping portion) can be made inconspicuous.

[0108]

According to the present invention, when a plurality of images including noise are combined (joined) to form one image (image signal), a smoothing process is performed on an overlapping portion of the combined images to form a link. It is possible to provide an image processing apparatus capable of making a matching portion (overlapping portion) inconspicuous.

[0109]

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram used for explaining a filtering process in the apparatus of FIG. 1;

FIG. 3 is a diagram used to describe a filtering process in another embodiment having the same configuration on the block diagram and different functions from the embodiment described with reference to FIGS. 1 and 2;

FIG. 4 is a block diagram showing still another embodiment of the present invention.

FIG. 5 is a diagram used for explaining a filtering process in the apparatus of FIG. 4;

FIG.

FIG. 7 is a diagram illustrating a filtering process according to another embodiment which has the same configuration on the block diagram as the embodiment described with reference to FIGS. 4 and 5 and different functions.

FIG. 8 is a block diagram showing still another embodiment of the present invention.

9 is a timing chart showing a timing relationship of signals (data) of respective units in the device of FIG.

FIG.

FIG. 11 is a diagram used for describing filtering processing in the embodiment described with reference to FIGS. 8 and 9;

FIG. 12 is a diagram used for explaining a filtering process in the device of FIG. 8;

FIG. 13 is a block diagram showing still another embodiment of the present invention.

FIG.

FIG. 15 is a diagram used for describing filtering processing in the embodiment described with reference to FIG. 9 using FIG.

FIG.

FIG. 17 is a diagram having the same configuration on the block diagram as the device in FIG. 13 and used for describing filtering processing in a device according to another embodiment having different functions.

FIG. 18 is a conceptual diagram illustrating a state in which a plurality of images to be subjected to a bonding process are generated by a plurality of image sensors.

19 is a schematic diagram showing a state of one continuous image obtained by laminating two images generated by the imaging method as shown in FIG. 18;

FIG. 20 is a block diagram showing a configuration of a conventional device.

FIG. 21 shows signals (data) of respective units in the apparatus shown in FIG. 20;
FIG. 4 is a timing chart showing the timing relationship of FIG.

FIG. 22 is a diagram illustrating characteristics of a smoothing process in the device described with reference to FIGS. 20 and 21.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Image sensor 11 Analog signal processing part 12 A / D converter 13 Memory 14 Multiplier 20 Image sensor 21 Analog signal processing part 22 A / D converter 23 Memory 24 Multiplier 30 Adder 40 Mixing ratio distributor 50 CPU 100 Filter 101 line memory 102 adder 110 filter 111 flip-flop 112 adder 200 filter 201 line memory 202 adder 210 filter 211 flip-flop 212 adder 300 adder 310 adder 320 adder 330 adder 400 selector 500 CPU 501 output switching command Part 600 Mixing ratio distributor 610 Multiplier 620 Multiplier 630 Multiplier 640 Multiplier

Claims (3)

[Claims] 1. An image processing apparatus for performing image data processing corresponding to an operation of combining a plurality of images with an overlapped portion and combining the images, the image data belonging to one image and the one image With respect to image data belonging to another image to be joined to the image data, image data corresponding to a portion other than the overlapping portion is subjected to an independent filtering process for each of the image data belonging to the one and the other images. The image processing apparatus further includes a smoothing unit configured to perform a filtering process on the corresponding image data based on both the image data belonging to the one and the other images so as to make the boundary of the joining unit inconspicuous. Image processing apparatus. 2. The overlapping portion is generated between vertically adjacent images by arranging a plurality of images in the vertical direction, and the smoothing means is provided for the plurality of images arranged in the vertical direction. Of the image data belonging to the corresponding one image and the image data belonging to another image to be connected to the one image, the image data corresponding to the portion other than the overlapping portion is included in the first and other images. The filtering process is independently performed on the image data of each of the horizontal lines belonging thereto, and the image data of each of the horizontal lines corresponding to the overlapping portion is determined based on both the image data belonging to the one and the other images. 2. The image processing apparatus according to claim 1, wherein the apparatus is configured to perform a filtering process for each line. 3. The overlapping portion is generated between horizontally adjacent images by arranging a plurality of images in a horizontal direction, and the smoothing means is provided for the plurality of images arranged in the horizontal direction. Of the image data belonging to the corresponding one image and the image data belonging to another image to be connected to the one image, the image data corresponding to the portion other than the overlapping portion is included in the first and other images. The filtering process is independently performed on the image data of each of the horizontal lines belonging thereto, and the image data of each of the horizontal lines corresponding to the overlapping portion is determined based on both the image data belonging to the one and the other images. 2. The image processing apparatus according to claim 1, wherein the apparatus is configured to perform a filtering process for each line.