JPH11183128A - Image processing device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To obtain depth coordinates of an image with high speed and high accuracy. A camera captures two images. Line segments are extracted from the image by edges. At this time, a line segment is divided into short lines by setting one line as a blank line for each predetermined line. The line segment is extracted, and the start point and end point coordinates are obtained. A combination in which the coordinate deviation in the direction (y-axis direction) perpendicular to the camera separation direction (x-axis direction) is within a predetermined range is extracted. Further, the start point and the end point of the same line are set to 1
As a set, a set in which the y-coordinate deviation is within a predetermined range for both the start point and the end point is defined as a corresponding line segment. In this way, the correspondence between each start point and each end point of each line segment can be obtained on a plurality of screens. If the correspondence is obtained, the deviation of the corresponding point in the x-axis direction is calculated. Using this deviation, the depth coordinate z of the point is determined.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to depth coordinates of each point on an image from an image obtained by imaging the environment with a plurality of cameras (from the image pickup position to the actual position corresponding to each point on the image). In particular, the present invention can be applied to a device for obtaining a three-dimensional image.

[0002]

2. Description of the Related Art Conventionally, as a method of obtaining depth coordinates of each point on a captured image plane, there is a stereo perspective method of obtaining depth coordinates using a plurality of images obtained by imaging an object with a plurality of cameras. In this method, the accuracy of the depth coordinates and the determination speed are determined based on the determination accuracy and the determination speed of the corresponding points on a plurality of screens. As a method of determining a corresponding point, a method of obtaining a block correlation in a plurality of gray images or a method of determining a corresponding point of an edge in a plurality of edge images is generally adopted.

[0003]

However, in the method using block correlation, it is necessary to correct a difference in luminance between two images, and an erroneous corresponding point is extracted unless a block is made large to some extent. However, on the contrary, there is a problem that if the block is too large, the positional accuracy of the corresponding point cannot be obtained. In addition, the method using an edge image has a problem that the corresponding points cannot be extracted unless a clear edge is obtained, and therefore the number of corresponding points that can be extracted is small.

The present invention has been made to solve the above-mentioned problem, and an object of the present invention is to increase the number of corresponding points extracted by increasing the number of areas extracted as edges. It is another object of the present invention to increase the number of corresponding points extracted to increase the number of candidates and to enable accurate extraction of corresponding points. Still another object is to accurately and quickly extract corresponding points.

[0005]

SUMMARY OF THE INVENTION The present invention calculates the depth coordinates of each point on an image by inputting video signals from a plurality of cameras and comparing each image obtained from the output of each camera. In the image processing device to input the video signal from each camera, to determine the edge intensity and edge angle determined from the gradient of the gray level of each pixel,
Edge pixels are determined from the edge intensity and the edge angle of each pixel, corrected edge data from which the display of the edge pixels is deleted is determined at least one row for every predetermined plurality of rows, and a line is determined based on the corrected edge data. A line segment extraction device that performs labeling every minute and obtains start point coordinates and end point coordinates for each line segment,
The line segment information memory that stores the start point coordinates and the end point coordinates of each line segment extracted by the line segment extraction device, and the start point coordinates and the end point coordinates of each line segment in each image are compared, and the corresponding start point or end point of each image is compared. A depth calculation device that calculates a position deviation of the plurality of cameras along the separation direction and calculates depth coordinates of each point on the image according to the position deviation.

[0006]

One method of comparing the start point or the end point of the extracted line segment between the plurality of images in the plurality of images described above is the direction in which the camera is separated (if the camera is installed horizontally, The combination in which the coordinate deviation in the direction perpendicular to the horizontal scanning line direction (x-axis direction) (when the camera is installed horizontally, the vertical operation line direction (y-axis direction)) is within a predetermined range is extracted. Further, a start point and an end point of the same line segment are set as one set, and a set in which the y coordinate deviation is within a predetermined range for both the start point and the end point is set as a corresponding line segment. At this time, the fact that the deviation of the edge angle in a plurality of screens of the line segment is within a predetermined value may be added to the corresponding condition.
If there are a plurality of extracted corresponding points, x
The one with the smallest coordinate deviation is taken as the corresponding point, and the x-coordinate deviation is smaller than the maximum deviation determined from the closest distance since the closest point from the camera is often known in the image. A point may be selected as a corresponding point. In this way, the correspondence between each start point and each end point of each line segment can be obtained on a plurality of screens. If the correspondence is obtained, the deviation of the corresponding point in the x-axis direction is calculated. Using this deviation, the depth coordinate z of the point is determined.

As described above, since a line segment is extracted and its start point and end point are compared, the correspondence between the line segments can be clearly grasped, and the corresponding start point and end point can be determined from the deviation in the x direction between the corresponding start point and end point. Since the depth coordinate z of the point can be calculated, the depth coordinate can be obtained at high speed because the calculation for obtaining the correspondence is simplified. Further, in a plurality of images, a line segment is detected from an edge, and furthermore, a start point and an end point of each line segment are detected, and a correspondence between the start point and the end point is detected. Moreover, since the deviation of the corresponding point in the x-axis direction can be obtained with high accuracy, the depth coordinates of each point can be obtained with high accuracy.

[0008] The term "separation direction of cameras" refers to a direction perpendicular to the optical axes on a plane formed by the optical axes of a plurality of cameras. In the image, the direction is the x axis, the direction perpendicular to the x axis is the y axis, and the optical axis direction, that is, the depth, is the z axis. When two cameras are installed horizontally and separated from each other on a horizontal plane, the horizontal scanning line direction on the image is the x-axis, and the vertical scanning line direction is the y-axis. However, the camera may not be installed horizontally or may not be installed on a horizontal plane. In those cases, depending on the attitude and arrangement of the camera,
As described above, the x-axis and the y-axis are set in the predetermined directions on the image.

In addition, in addition to the above-described method, a method of detecting a corresponding point in a plurality of images is a method for detecting a grayscale image of a plurality of pixels in the x-axis direction starting from a start point or an end point of a line segment. The correlation may be obtained, and the correspondence that best matches from the correlation value may be detected. Alternatively, a correlation may be obtained with respect to a gray-scale image in the x-axis direction between corresponding line segments obtained from a plurality of images, and a correspondence that best matches the correlation may be detected from the correlation value.

[0010]

FIG. 1 shows an embodiment of the present invention. In FIG. 1, an environment is imaged by a camera 1,
Is input to the line segment extraction device A1. The line segment extraction device A1 includes an A / D converter 12, a filter processing circuit 13, an edge line extraction circuit 14, a labeling device 1,
5, a label image memory 16, a line segment information memory 17, and a data correction device 18. A / D converter 12
Is converted into a digital image. The digital image is filtered (eg, a Sobel filter) to calculate edge angle data and edge strength data. The edge angle data and edge strength data are input to the ridge line extraction circuit 14. The ridge line extraction circuit 14 is a circuit that determines whether each pixel is a pixel on the ridge line. The ridge line data output from the ridge line extraction circuit 14 is data including a ridge line flag indicating that each pixel is a ridge line and edge angle data. The edge data is input to the data correction device 18 in synchronization with the horizontal synchronization signal and the pixel synchronization signal. In the data correcting device 18, the corrected edge data from which at least one edge data is deleted every predetermined plurality of rows is input to the labeling device 15. In the labeling device 15, for each pixel, the label value of the line segment where the pixel exists and the pixel address of the start pixel and the end pixel of each line segment (the horizontal scanning line direction is the x axis, the vertical scanning line direction is the y axis, (X coordinate, y
Coordinates)) are determined. The results are stored in the label image memory 16 and the line segment information memory 17, respectively.
Such a line segment extraction device A2 is also provided for another camera 2. The environment is photographed in stereo by the camera 1 and the camera 2. Then, based on the pixel addresses of the start pixel and the end pixel of the line segment in each of the images extracted by the line segment extraction devices A1 and A2, the depth calculation device 20 causes the depth coordinate z (point of the camera) at each point on the image. Taking the z-axis in the optical axis direction, the distance from the camera position to the real image point) is calculated.

FIG. 2 shows the configuration of the edge line extraction circuit 14. The ridge line extraction circuit 14 uses the edge angle data and the edge strength data as one set of input data. Line delays 21 and 2 for delaying data by one line of an image
2 are arranged in series in two stages. With this configuration, it is possible to output the edge angle data and the edge strength data for three lines, including the data of the current line, the data of one line before and the data of two lines before, to the shift register circuit 23 at a time.

The shift register circuit 23 sequentially inputs input data for three lines and stores 3 × 3 pixel edge angle data and edge intensity data corresponding to a 3 × 3 matrix on the screen. It comprises nine registers R _1,1 , R _1,2 ... R _3,3 arranged in a matrix. The number of bits in each register is the number of bits that can store edge angle data and edge strength data. The data of the target pixel is stored in the central register _R2,2 .

The flag setting circuit 24 is a circuit for setting an edge flag and an edge strength flag from 3 × 3 matrix data set in the shift register circuit 23. From the data of the target pixel and the surrounding eight pixels, it is determined whether or not the target pixel is a ridge line pixel, and whether or not the edge strength of the target pixel is equal to or greater than a threshold value.
Set or reset the edge flag and the edge strength flag. The edge line flag indicates that, in a direction perpendicular to the edge angle of the pixel of interest, the edge intensity of the pixel of interest is greater than the edge intensity of two neighboring pixels adjacent to the pixel of interest, and the edge intensity is greater than the threshold Vth1. Is also set if Hereinafter, a pixel for which the edge flag is set is referred to as an edge pixel. The edge strength flag is set if the edge strength of the target pixel is larger than a certain threshold value Vth2. At this time, the threshold value Vth2 is
A larger value.

The effect of giving two thresholds to the edge strength as described above is that the edge flag can be extended as a ridge pixel even with a small edge strength as much as possible so that the line segment can be connected without interruption. On the other hand, if a line segment is composed of only small edge intensities, that is, if the edge intensity flag is not set for all the edge pixels of one line, it is later recognized that the reliability of the line segment is low. If the edge strength flag is set on at least one ridge line pixel in one line segment, this line segment can be set as a highly reliable line segment.

The flag setting circuit 24 corrects the ridge line data including the ridge line flag F1, the edge strength flag F2, and the edge angle data G with respect to the target pixel in synchronization with the input of data for each pixel from the filter processing circuit 13. Output to the device 18.

FIG. 3 shows the configuration of the data correction device 18. The counter circuit 181 is a circuit that updates a numerical value each time a horizontal synchronization signal is input, and the current row number can be determined from the stored value of the counter circuit 181. The row number is input to the erase row designation circuit 182. The erase row designating circuit 182 is configured by, for example, a circuit that detects the rise of the signal of the lower fourth digit of the counter circuit 181, so that the erase row designating signal is output to the data conversion circuit 183 every eight rows. it can. The data conversion circuit 183 receives ridge line data from the ridge line extraction circuit 14 and sets the ridge line flag of the ridge line data of the row to 0 (1 is a pixel on the ridge line, 0 Is defined as a pixel outside the ridge line). In this way, it is possible to delete one line of edge data every eight lines. At least one deleted line is sufficient, and a plurality of deleted lines may be used. The number of lines to be deleted is optional.

FIG. 4 shows the configuration of the labeling device 15. The labeling device 15 comprises a line delay 31 for delaying data by one line, and two stages and five shift registers S _1,1, S _1,2, S _1,3, S _2,1, S _2,2. It has a shift register circuit 32, a label image memory 33 for storing a label value for each pixel on the screen, and a shift register circuit for referring to or storing the label values of peripheral pixels.

The input data is the corrected ridge line data output from the data correcting device 18 that includes a set of the ridge line flag F1, the edge strength flag F2, and the edge angle data G. 1
The data of the current line is input to the first-stage shift registers S _1,1, S _1,2, S _1,3 , and the second-stage shift register S
Data delayed by one line is input to _{2,1, S2,2} . The correspondence between the shift register circuit 32 and the pixels on the screen is as shown in FIG. The scanning direction of the pixels is from left to right on the screen and from top to bottom. FIG. 4 shows the correspondence between the data of each pixel in the window region T and the shift register circuit 32.

The shift register circuit 34 is provided with W _{1,1 to} W _1,4 for the current pixel and W _{2,1 to} W _2,4 for the previous pixel. The shift register circuit 34 is for holding the label value corresponding to each pixel of the shift register circuit 32. Therefore, in principle, the number may be two for the previous row and three for the current row. The reason why the same number is provided is to store the label values of the pixels in the same column in the label image memory 33 at the same time, and the extra one is a buffer for saving the data processing time. The values read out from the label image memory 33 are stored in the shift registers W _{2,1 to} W _2,4 corresponding to the pixels in the preceding row, and new label values are set. Shift registers W _{1,1 to} W _1,4 corresponding to the current pixel
Is set to a newly determined label value. Each label value set in the shift register circuit 34 is sequentially stored in the label image memory 33. As described above, the shift operation and data setting of the shift register circuits 32 and 34 are performed based on the synchronization signal output from the control circuit 36.
This is executed in synchronization with the pixel column.

The label value assigning circuit 3 of the labeling device 15
7 inputs the values of the shift register circuits 32 and 34,
The determined label value is set in the shift register circuit 34. As shown in FIG. 5, a labeling process is performed from five sets of data including a target pixel and four peripheral pixels. In the labeling process, in addition to the ridge line flag F1, the edge strength flag F2, and the edge angle G of the target pixel and the four surrounding pixels, a label image that stores the label value is necessary because the label value that has already been added is required. The label value is read from the memory 33 in synchronization with the input of the data such as the edge line flag and the like, and used for the labeling process.

The basic algorithm of the labeling process executed by the label value providing circuit 37 will be described below. If the pixel of interest is not an edge pixel with no edge flag set, the process proceeds to the next pixel of interest without doing anything. When the target pixel is a ridgeline pixel, each process is performed in the following cases.

1) If the target pixel has not been labeled yet, and if the four peripheral pixels are not the edge pixels with the edge flag set, the target pixel is an isolated point and no label value is given.

2) If the target pixel has not been labeled yet, if at least one of the four peripheral pixels is an edge pixel and none of the peripheral edge pixels is labeled, With the target pixel as the starting point, the same label is assigned to the target pixel and the edge pixels of the peripheral pixels. At this time, when examining the peripheral edge pixel, it is determined whether or not the difference between the edge angle of the pixel of interest and the edge angle of the peripheral edge pixel is within a certain range, and whether or not the edge is the same line segment. A judgment process is added. This processing is not particularly required.

3) When the target pixel has not been labeled yet, and at least one of the four peripheral pixels is an edge pixel and any of the edge pixels is already labeled, The same label value as the label already attached to the unlabeled edge pixel in the target pixel and peripheral edge pixels is assigned.

4) If the target pixel has already been labeled and none of the four peripheral pixels is an edge pixel, the end point of the line segment is determined.

5) If the target pixel is already labeled, and if at least one of the four peripheral pixels is an unlabeled edge pixel, the same label is assigned to the edge pixel.

The address of the pixel determined to be the start point in the process 2) is the address of the pixel determined to be the end point in the process 4). It is stored in the minute information memory 35. The line segment information memory 35 also stores the edge angles of the start point and the end point. In the process 2), the start point is determined. In this case, a new line segment is started. Therefore, every time a new start point is detected, a label value of a line segment whose counter value is updated by 1 can be assigned.

By implementing the above basic algorithm with the hardware of the label value assigning circuit 37, the labeling process can be performed in synchronization with the timing clock of the video signal input from the camera. Pipelining can be performed up to labeling processing, and can be realized with a delay of several lines of image data from input to completion of labeling processing. As a result, a labeling process suitable for real-time image processing of an automobile or the like can be realized.

In the labeling process of the present system, the above-described apparatus is an algorithm for inputting image data for two lines and extracting line segments based on the image data. For this reason, there is no problem in particular with respect to a line segment that descends to the right, but there is a problem that the same line segment is divided for a line that rises to the right (particularly, a line that rises to the right with a small inclination). Therefore, if a line segment that ascends to the right is cut off, the line segment that should originally have the same label is cut off with another label, so that these two line segments are the same line segment. By storing another label value as a continuous pointer in the label information of a label attached later, the label can be treated as the same label in a pseudo manner.

When the edge strength flag is set for the pixel of interest, the label value assigning circuit 37 sets a reliability flag in the area corresponding to the label value of the pixel of interest in the line segment information memory 35. ing. At the end of one-screen labeling, the line segment with the reliability flag set
At least one pixel on the line segment means that the edge intensity is equal to or higher than the threshold value Vth2, which means that the line segment can be sufficiently used as a line segment. Conversely, a line segment for which the reliability flag is not set means that the edge strength is weak and cannot be adopted as a line segment, and this information can be used in post-processing.

Since the ridgeline flag is set to 0 in the deleted line every eighth line by the data correcting device 18, the contents of the label image memory 16 which is the result of labeling for each line segment by the labeling device 15 are as follows. As shown in FIG. The numerical values in FIG. 6 are label numbers assigned to each continuous line segment. As shown in FIG. 6, line segment labeling is performed for each of the divided areas divided by the erase row (eighth row). Therefore, in the case of a road white line, a pair of parallel ridge lines is detected. The recognition of the parallel ridge line is based on the fact that the label numbers of the two line segments are close to each other, whether the start point and the end point of the two line segments are in the adjacent line of the erased line, or in the vicinity of the adjacent line. The determination can be made by comprehensively considering whether or not the edge angles of the line segments are parallel.

In FIG. 6, a pair of ridge line segments of the white line is
It is a set of label 1 and label 2, and a set of label 5 and label 6. FIG. 7 showing the contents stored in the line segment information memory 17.
As shown in the label 1, the start point coordinates (x _s, y _s) of the line segment of the label 2, respectively, (5,1), (8,1)
And the end point coordinates (x _e , y _e ) are (7,
7), (10, 7). The segment information start, end points of the edge angle (f _s, f _e) Since also included, and the 1
It can be determined whether or not the paired line segments are parallel, that is, whether or not the line segment is a white line of the recognition target.

The start point coordinates (X _s , Y _s ), end point coordinates (X _e , Y _e ), and edge angle (F) of the line segment imaged by the camera 2 and processed by the line segment extraction device A 2 are obtained. _s , F _e ) are obtained as shown in FIG. In the coordinate display, lowercase letters indicate values in the first image obtained by the camera 1 and uppercase letters indicate values in the second image obtained by the camera 2. Based on the line segment information obtained from the two images shown in FIGS. 7 and 8, the depth calculation device 20 calculates the depth coordinate z (from the imaging point to the actual position corresponding to the point on the screen) as follows. Is calculated. As shown in FIG. 9, the distance to the object is z, the number of pixels in one line in the x-axis direction of the screen is N, and the viewing angle of the camera corresponding to one line of the screen is 2θ. Actual horizontal distance 2ztan θ
Corresponds to one line. Therefore, 2z tan θ / N is the actual horizontal distance per pixel. The shift amount between the corresponding points P1 and P2 in the first image and the second image is defined as Δx (the number of pixels). The actual horizontal length corresponding to the shift amount Δx is equal to the actual distance L between the camera 1 and the camera 2.

Therefore,

L = 2zΔxtan θ / N (1) Therefore, the depth coordinate z of the target point is

## EQU2 ## z = LN / 2Δxtan θ (2) As described above, if the shift amount Δx of the corresponding point on the two images on the screen is obtained, the depth coordinate z of the point can be obtained.

The shift amount Δx can be obtained as follows. The depth calculation device 20 is constituted by a computer system, and FIG. 10 is a flowchart showing a processing procedure of the CPU. First, in step 100, the label number A of the line segment in the first image is initialized to 1.
Next, in step 102, line segment information of the label number A is extracted from the first image. The line segment information is
Start point coordinates (x _As , y _As ), end point coordinates (x _Ae , y _Ae ), and edge angles (f _As , f _Ae ) between the start point and the end point.

Next, in step 104, the start point coordinates (x _As , y _As ) and the end point coordinates (x _Ae , y _Ae ) of the extracted label number A are both set in the second image with respect to the start point and the end point. A line segment whose y coordinate has a deviation within ± 1 is extracted. 7 and 8, since the y coordinate of the start point of the line segment of A = 1 is 1 and the y coordinate of the end point is 7,
The line segment in the second image satisfying the condition is labeled 1,
2, 3, and 4.

Next, from the candidate line segments of the second image selected in step 104, the edge angle of the line segment (f _As , f _Ae ) with respect to the line angle of the label value A of the first image (f _As , f _Ae ) A line segment whose deviation of F _Ms , F _Me ) is equal to or less than a predetermined value (for example, ± 21 degrees) is extracted. In the second image, the line segment having the label values 1 and 3 is extracted as corresponding to the line segment having the label value 1 of the image in the first image.

Next, in step 108, among the candidate image segments of the second image narrowed down to step 106, the X-coordinates X _Ms and X _{Me of the} start point and the end point are changed to the line of the label value A of the first image. Those having a deviation of a predetermined value or less from the x-coordinates x _As and x _Ae of the start point and the end point of the minute are further determined as candidates. The predetermined value Δx _max at this time is determined by the following equation obtained by modifying equation (2).

[0039]

Δx _max = LN / 2z _min tan θ (3) Here, the distance L between the origins of the cameras 1 and 2, the number N of pixels of one line on the image, the viewing angle 2θ of the camera, and the It is assumed that the closest distance z _min from the camera is known. With these values, the depth coordinate z of all points on the image is larger than the closest distance z _min , and the shift amount Δx of the corresponding point in the x-axis direction is larger than Δx _max determined by Expression (3). Does not grow. When the camera 2 is provided at a distance L from the camera 1 in the positive direction of the x-axis, the second
The corresponding point of the image is displaced in the negative direction of the x-axis in comparison with the first image. Therefore, from among the candidate line segments of the second image, 0
≦ x _As −X _Ms ≦ Δx _max and 0 ≦ x _Ae −X _Me ≦ Δx
_Line segments satisfying _max are extracted. For example, N = 500,
If L = 0.2 m, θ = 15.4 °, and z _min = 5 m, then Δx _max = 37. In FIG. 8, the candidate line segment is 1,
Although it is 3, the line segment satisfying the above condition is 1. still,
If a plurality of line segments correspond, the one with the smaller deviation is extracted as the last corresponding line segment. Thus, the line segment of the second image corresponding to the label value A of the first image is extracted.

Next, in step 110, it is determined whether or not the label value A is final. If not, in step 112, the label value A is incremented by one, and the process returns to step 102 to return to the first image. The second corresponding to the next line segment
Extraction processing is performed on the line segments of the image.

If the correspondence relation is obtained for all the extracted line segments, in step 114, the deviation Δx of the x coordinate is calculated for each start point and end point of each corresponding line segment. And
The depth coordinate z of the point is calculated by equation (2). In this manner, in the first image, the depth coordinate z at each of the start point and the end point of each line segment can be obtained. As a result, the (x, y, z) three-dimensional coordinates of the start point and the end point are determined, so that a three-dimensional image can be obtained.

As described above, according to the present invention, in the image, the line segment is forcibly divided in the y-axis direction, so that the line segment does not become unnecessarily long. As a result, the shift amount Δx of the x coordinate at the start point and the end point becomes substantially the same, so that the corresponding point search becomes easy.

In addition to the above embodiment, a line segment extraction device A
1, a frame memory 19 for storing a grayscale image output from the A / D converter 12 may be provided in A2 to accumulate data of one frame. Then, when corresponding points are detected in the two images, the above-described step 106 is performed.
After extracting the corresponding line segment by the processing up to, the starting point of the line segment,
Alternatively, a correlation may be obtained with respect to a grayscale image of a plurality of pixels in the x-axis direction starting from the end point, and a correspondence that best matches the correlation may be detected from the correlation value. Alternatively, a correlation may be obtained with respect to the grayscale image in the x-axis direction between the corresponding line segments obtained from the two images, and the most corresponding correspondence may be detected from the correlation value.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a ridge line extraction circuit of the embodiment device.

FIG. 3 is a block diagram showing a configuration of a data correction device of the embodiment device.

FIG. 4 is a block diagram showing a configuration of a labeling device of the embodiment device.

FIG. 5 is an explanatory diagram showing the correspondence between pixels on a screen and labeling processing pixels.

FIG. 6 is an explanatory diagram showing a result of labeling.

FIG. 7 is an explanatory diagram showing line segment information of a labeled line segment of a first image.

FIG. 8 is an explanatory diagram showing line segment information of a labeled line segment of a second image.

FIG. 9 is an explanatory diagram showing a method of calculating depth coordinates from corresponding points of two images.

FIG. 10 is a flowchart illustrating a processing procedure of a CPU included in the depth coordinate calculation device.

[Explanation of symbols]

 1, 2 ... Camera A1, A2 ... line segment extraction device 15 ... labeling device 18 ... data correction device 31 ... line delay 32, 34 ... shift register circuit 33 ... label image memory 35 ... line segment information memory 37 ... label value assignment circuit 20 depth coordinate calculation device

Claims (1)

[Claims] An image processing apparatus which receives video signals from a plurality of cameras and compares respective images obtained from the outputs of the respective cameras to calculate depth coordinates of respective points on the images. From the input, the edge intensity and the edge angle determined from the gradient of the gray level of each pixel is determined, the ridge pixel is determined from the edge intensity and the edge angle of each pixel, every predetermined plurality of rows Line segment extraction in which corrected edge data from which the display of edge line pixels has been deleted is determined at least in one line, labeling is performed for each line segment based on the corrected edge line data, and start point coordinates and end point coordinates are determined for each line segment. A line segment information memory that stores the start point coordinates and the end point coordinates of each line segment extracted by the line segment extraction device; and the start point coordinates and the end line of each line segment in each image. Depth calculation for comparing the coordinates, calculating the position deviation of the corresponding start point or end point in each image along the direction away from the plurality of cameras, and calculating the depth coordinates of each point on the image according to the position deviation. An image processing device comprising: a device;