JPH11261868A - Fisheye lens camera apparatus, image distortion correction method thereof, and image extraction method - Google Patents

Abstract

(57) Abstract: A fisheye lens camera apparatus, an image distortion correction method thereof, and an image extraction method, which speed up image deformation processing for distortion correction of a fisheye lens image when the fisheye lens camera is mounted at an arbitrary installation angle, and , A moving image such as a human image is accurately detected and extracted, and displayed on a monitor television or the like with high accuracy. SOLUTION: Fisheye lens 1-1, CCD imaging device 1-
The image photographed in Step 2 is stored in the video memory 1-3, and the image correction processing unit 1-4 performs an arithmetic operation by combining the coordinate conversion for correcting the installation angle of the fisheye lens camera and the coordinate conversion for correcting distortion of the fisheye lens image. And transforms a fisheye lens image of equal area projection at high speed. Further, a configuration is provided in which weighting corresponding to the area in the fisheye lens image is performed to extract a feature amount such as a human image, and a display area is extracted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fish-eye lens camera apparatus, an image distortion correcting method thereof, and an image extracting method, and more particularly to a video conference in which an image is captured by a fish-eye lens camera, a part of the image is cut out, and displayed on a monitor television. The present invention relates to a fish-eye lens camera device in a system or a remote monitoring system, and a method for correcting image distortion and an image extracting method.

A fish-eye lens has an angle of view of about 180 degrees and can display a wide range of images. However, the images are distorted in a barrel shape, and in particular, the peripheral portion is significantly distorted. The present invention makes use of the fact that the imaging range of a fisheye lens is wide, cuts out a portion of an image of interest from an image captured by one fisheye lens camera, corrects the image distortion, and displays the image on a monitor TV screen. Is what you do.

Further, the present invention detects an image accompanied by movement such as intrusion of a person from an image projected by a fish-eye lens camera, and automatically determines a monitor display area (cutout area) of the image. The moving image is tracked, and is suitably used for a camera device of a remote monitoring system or a video conference system.

[0004]

2. Description of the Related Art FIG. 13 is a diagram showing a configuration of a conventional fish-eye lens camera device and a remote monitoring system and a video conference system using the same. FIG. 1A shows the configuration of a fish-eye lens camera device, FIG. 2B shows a remote monitoring system using the fish-eye lens camera device, and FIG. 1C shows a video conference system using the fish-eye lens camera device.

In FIG. 1, reference numeral 13-10 denotes a fish-eye lens camera device, 13-1 denotes a fish-eye lens, 13-2 denotes a CCD image pickup device, 13-3 denotes a video memory (frame memory),
3-4 is an image correction processing circuit, and 13-5 is a corrected image (NT
SC) An output circuit.

The fish-eye lens 13-1 optically projects an object at an angle of view of about 180 degrees, and the CCD imaging device 13-2 generates an electric image signal from an optical image, and a video memory (frame memory). 13-3 stores the image signal in frame units.

[0007] The image correction processing circuit 13-4 corrects the distorted image of the fish-eye lens for the monitor display area to restore the original image, and outputs a corrected video (NTSC) output circuit 13-5.
Outputs the corrected image signal as a normal television video signal of the NTSC system or the like.

Since the angle of view of a fisheye lens is wide, a remote monitoring system or a video conference system that captures images of a wide area by using a camera device using the lens can be configured. As shown in (b) and (c) of the drawing, the encoding device 13-20 is attached to the fish-eye lens camera device 13-10, or the fish-eye lens camera device 1 with the encoding device.
According to 3-30, the video signal is transmitted via the public line 13-40 and displayed on the monitor television screen 13-60 or the like via the decoding device 13-50, whereby those systems can be configured. . However, since the fisheye lens has a wide angle of view, the image distortion is large, and the distortion must be corrected and displayed on the monitor television screen 13-60 or the like.

FIG. 14 is an explanatory diagram of distortion correction of a conventional fish-eye lens camera image. In the figure, 14-1 is a virtual image frame, 14-2 is a virtual hemisphere, and 14-3 is a fisheye lens imaging screen. In addition, 3 shown by x, y, z axes
It is assumed that the fisheye lens is placed at the origin O of the dimensional space, and that the fisheye lens is placed in the z-axis direction.

The virtual hemispherical surface 14-2 is a virtual hemispherical spherical surface having a radius f when the fish-eye lens is an orthographic projection lens and its focal length is f. It is placed on the y-plane, and its center is placed at the position of the origin O.

The virtual image frame 14-1 is a frame on a virtual plane orthogonal to a line of sight vector DOV (Direction Of View) from the position (origin) of the fisheye lens toward the object to be photographed, and has a predetermined size. As will be described later, the image in the frame is cut out from the fisheye lens image and displayed on the monitor television screen. That is, the frame has the same size as the frame of the display frame of the monitor television or the like.

The image frame 14-1 has a two-dimensional coordinate axis (p, q) having an origin at a point intersecting with the line-of-sight direction vector DOV within the frame.
The point in 4-1 is represented by the component (p, q) of this coordinate.

An image projected by the fish-eye lens (fish-eye lens image) is formed by a projection method of the lens (projection fomu).
la) can be divided into several types.
The fisheye lens of the orthographic projection system will be described.

It is assumed that an image viewed from the position (origin O) where the fisheye lens is placed through the virtual hemisphere 14-2 is directly attached to the virtual hemisphere 14-2. Then, the image of the three-dimensional space pasted on the hemisphere 14-2 is placed on a plane of the two-dimensional space composed of the x-axis and the y-axis in the figure, perpendicularly to the plane (from the z direction to the origin). The image crushed and attached is the image of the fisheye lens of the orthographic projection system.

In practice, the image of the fisheye lens forms an image at a position where the image is inverted upside down, left and right, but since the relative relationship of the image distortion does not change, the image is formed as described above to simplify the explanation. Shall be.

A plane in a two-dimensional space obtained by vertically crushing the image on the hemisphere 14-2 is a fisheye lens imaging screen 14-3, and an image inside a circle of the fisheye lens imaging screen 14-3 is taken with a fisheye lens. Image. The outer square frame is the outer frame of the entire imaging screen obtained from the CCD imaging device.

Since the image of the fisheye lens is distorted, it is necessary to cut out a part of the image and correct it to a normal image in order to display the image taken by the fisheye lens camera. Therefore, a frame corresponding to the frame to be displayed is assumed to be a virtual image frame 14-1.

It is assumed that the direction of the z-axis to which the fisheye lens is directed is a direction above the vertical line, and the line-of-sight direction vector D
The vertex angle (zenith angle) formed by the OV with the z-axis is φ, and the azimuth formed by the horizontal reference axis (x-axis or y-axis) is θ. The focal length (the radius of the circle of the fisheye lens image) is f, the rotation angle of the image frame is ω, and the magnification is m. The magnification factor m is m = D / f, where D is the distance from the position of the fisheye lens (origin O) to the origin in the image frame 14-1.

The point (x,
The correspondence between y) and the point (p, q) in the image frame 14-1 is represented by the relational expression of Expression (1).

[0020]

(Equation 1)

The distortion of the fish-eye lens image can be corrected by using the correspondence relationship of equation (1), and the original image can be restored and displayed on the monitor television screen. That is, first, the image frame is localized at 14-1, and an area (cutout area) to be displayed on the monitor is determined. This localization operation is set by inputting the vertex angle φ, the azimuth angle θ, the rotation angle φ, and the enlargement factor m of the image frame from the outside to the image correction processing circuit 13-4.

The image correction processing circuit 13-4 calculates the point (x, y) in the fisheye lens image plane 14-3 corresponding to the point (p, q) in the image frame 14-1 by the above equation (1). The color information signal at that point is read out from the video memory (frame memory) 13-3 by the relational expression, and
The color information signal is written into an output video memory (not shown) having an address corresponding to the point (p, q) in 4-1.

This color information transfer operation between the video memories is performed for all points (p, q) in the image frame 14-1, and the color information signals are sequentially output from the output video memory to the corrected video (NTSC) output. By sending the corrected image (NTSC) output circuit 13-5 to the circuit 13-5, the origin O
, A real image viewed through the frame of the virtual image frame 14-1 can be displayed without distortion. (See US Pat. No. 5,185,667)

[0024]

The above-described distortion correction of a fisheye lens image is a correction when a target fisheye lens is an orthographically projected image, and is different from an image of an equal area mapping which is often used as an actual fisheye lens. In the case of a fisheye lens,
The operation of the coordinate transformation becomes more complicated.

The calculation of the distortion correction of the fisheye lens image requires a huge amount of calculation because the coordinate conversion is performed for each coordinate point of the image frame to be displayed, and the calculation amount is further increased by increasing the resolution of the image. Therefore, the amount of calculation for coordinate transformation must be as small as possible.

The above-described distortion correction of the fisheye lens image is performed by:
Since it is assumed that the fish-eye lens camera is installed on a desk or the like and oriented directly upward in the ceiling direction, or mounted on the ceiling and oriented directly downward in the floor direction, it is supposed to be placed in the vertical direction,
The installation angle of the fisheye lens camera cannot be freely selected and installed. On the other hand, mounting the fisheye lens camera sideways on a wall or mounting it at an oblique angle to the corner of the ceiling can often capture an imaging area such as a monitoring area into a more effective imaging area of a fisheye lens image.

Furthermore, the image captured using a fish-eye lens camera actually varies depending on the characteristics of each fish-eye lens camera. The difference in the characteristics of the fish-eye lens camera is such that even if image conversion is performed by correcting the distortion of the fish-eye lens image, the converted image remains displayed as image distortion.

Conventionally, the correction of the horizontal displacement of the fisheye lens image on the image pickup screen of the CCD image pickup device and the correction of the aspect ratio caused by the structure of the CCD image pickup device, which are image parameters of the fisheye lens camera, must be manually performed. Had to adjust.

In a tracking type remote monitoring system using a fish-eye lens camera, detection of a human image and the like and determination of a monitor display area (cutout area) must be automatically performed. In the method of determining the monitor display area (cutout area) by using, the detection when there is only one moving object is the limit,
When a plurality of objects move in an area, it is impossible to create an image that tracks each of them.

Further, since the brightness and distortion of the fisheye lens image are different between the central portion and the peripheral portion, the image displayed on the monitor becomes unnatural as the cut-out region at the time of tracking changes. There was a drawback that it was easy to become.

The present invention speeds up image deformation processing for distortion correction of a fish-eye lens image when the fish-eye lens camera is mounted at an arbitrary installation angle, and also enables accurate movement of a moving image such as a human image taken by the fish-eye lens camera. An object is to detect and extract well and display it on a monitor television or the like with high accuracy.

[0032]

According to the present invention, there is provided a fish-eye lens camera device comprising: (1) a fish-eye lens camera device having an image correction processing unit for correcting distortion of an image photographed by the fish-eye lens camera; And a coordinate conversion for correcting the installation angle of the fisheye lens camera and a coordinate conversion for correcting distortion of the fisheye lens image. (2) The image correction processing section has a configuration for calculating coordinate transformation for correcting distortion of a fisheye lens image of an equal area mapping.

(3) In a fish-eye lens camera device provided with an image correction processing unit for correcting distortion of an image taken by a fish-eye lens camera, the center position of the display image and the position of the fish-eye lens image for distortion correction of the fish-eye lens image. It has a configuration in which parameters such as the aspect ratio and the radius of the fisheye lens image area are extracted from the captured fisheye lens image itself, and distortion correction of the fisheye lens image is performed using the parameters.

(4) In a fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, areas are integrated using mutual positional relationships in a fish-eye lens image, and the area where the image has changed is determined. It has a configuration for extracting.

(5) In a fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, the fish-eye lens image is converted into a panoramic landscape image, and a signal of a difference between frames of the image or an inter-frame difference is output. It is provided with a configuration in which a signal obtained by combining the difference and the intra-frame difference is extracted, an area is extracted using the signal, and areas of a plurality of moving images are simultaneously extracted.

(6) In a fish-eye lens camera device for extracting a moving image from an image captured by a fish-eye lens camera, a characteristic amount such as a signal of an inter-frame difference or a signal obtained by combining an inter-frame difference and an intra-frame difference is used. The extraction operation is performed based on the image of the fisheye lens image, and in the area extraction processing for the feature detection area, the address conversion between polar coordinates and orthogonal coordinates is performed to scan the inside of the image of the fisheye lens image. From a plurality of moving images.

(7) In a fisheye lens camera apparatus for extracting a moving image from an image taken by a fisheye lens camera, the shape of each human image is determined for a plurality of human image regions obtained from the fisheye lens image. Normalization is performed, color information in the shape of each image is stored, and an image is created that individually tracks a plurality of moving human images based on the color information of each image.

(8) In a fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, different weighting factors are multiplied according to the magnitude of distortion between the central portion and the peripheral portion of the fish-eye lens image. It is provided with a configuration for performing region extraction based on the feature amount. or,
(9) A configuration is provided in which different values are further given as the weighting factors corresponding to the regions in the image of the fisheye lens image, and the non-extracted regions are masked to perform region extraction.

The image distortion correcting method for a fish-eye lens camera according to the present invention comprises the steps of: (10) correcting the distortion of an image photographed by the fish-eye lens camera; This includes a process of performing an operation in combination with the coordinate transformation to be corrected. Further, (11) the image distortion correction of the fisheye lens camera includes a step of calculating coordinate transformation for correcting distortion of the fisheye lens image of the equal area mapping.

(12) In correcting the distortion of the image taken by the fish-eye lens camera, parameters such as the center position of the display image, the aspect ratio of the fish-eye lens image, and the radius of the fish-eye lens image area for correcting the distortion of the fish-eye lens image. Is extracted from the captured fisheye lens image itself, and distortion correction of the fisheye lens image is performed based on the extracted parameters.

The image extraction method for a fish-eye lens camera according to the present invention is characterized in that (13) an image extraction method for extracting a moving image from an image photographed by a fish-eye lens camera by utilizing a mutual positional relationship in a fish-eye lens image. It includes a process of integrating regions and extracting a region where an image has changed.

(14) In an image extracting method for extracting a moving image from an image taken by a fish-eye lens camera, a fish-eye lens image is converted into a panoramic landscape image, and a signal of a difference between frames of the image or an inter-frame difference is output. The method includes a process of extracting a signal obtained by combining the difference and the intra-frame difference, extracting a region based on the signal, and simultaneously extracting regions of a plurality of moving images.

(15) In an image extracting method for extracting a moving image from an image taken by a fish-eye lens camera, a feature amount such as a signal of an inter-frame difference or a signal combining an inter-frame difference and an intra-frame difference is used. The extraction operation is performed based on the image of the fisheye lens image, and in the area extraction processing for the feature detection area, the address conversion between polar coordinates and orthogonal coordinates is performed to scan the inside of the image of the fisheye lens image. From a plurality of moving images.

(16) In an image extraction method for extracting a moving image from an image taken by a fish-eye lens camera, the shape of each human image is determined for a plurality of human image regions obtained from the fish-eye lens image. It involves normalizing, storing color information in the shape of each image, and creating a video that individually tracks a plurality of moving human images based on the color information of each image.

(17) In an image extraction method for extracting a moving image from an image taken by a fish-eye lens camera, different weighting factors are multiplied according to the magnitude of distortion between a central portion and a peripheral portion of the fish-eye lens image. This includes a process of extracting a region based on the feature amount.

(18) The method further includes a step of giving a different value as the weighting factor corresponding to a region in the image of the fisheye lens image, and masking a non-extracted region to perform region extraction.

[0047]

FIG. 1 is a diagram showing the configuration of a fish-eye lens camera device according to an embodiment of the present invention. In the figure, 1-1 is a fisheye lens, 1-2 is a CCD imaging device, 1
-3 is a video memory (frame memory), 1-4 is an image correction processing unit, 1-41 is a camera installation angle correction unit, 1-42 is a pan / tilt angle rotation correction unit, 1-43 is a zoom correction unit,
1-44 is an image interpolation creating unit, and 1-5 is a corrected image (NTS
C) An output circuit.

A wide-range image is optically captured by the fish-eye lens 1-1, an electric image signal is generated from the optical image by the CCD image pickup device 1-2, and the image memory (frame memory) 1-3 stores the image. The signal is stored in frame units.

The image correction processing section 1-4 corrects the image distorted by the fish-eye lens and restores the original image, and the corrected video (NTSC) output circuit 1-5 converts the corrected image signal into a normal image signal. As a TV video signal of the NTSC system or the like. Also, this signal can be encoded by an encoding device and transmitted, and the image can be displayed at a remote location.

The camera installation angle correction section 1-41 of the image correction processing section 1-4 corrects the installation angle of the fish-eye lens camera device facing an arbitrary direction, and the pan / tilt angle rotation correction section 1-4.
Numeral 2 controls the amount of rotation of the pan / tilt angle of the video display area (cropping area), the zoom correction unit 1-43 controls the enlargement ratio of the display video, and the video interpolation creation unit 1-44 controls the coordinate point. Is performed to correct the distortion of the image in the video display area (cutout area) and return to the original image.

FIG. 2 is an explanatory diagram of an image photographed by the fisheye lens camera according to the embodiment of the present invention. (A) of FIG.
The fisheye lens image img when the fisheye lens camera is pointed directly above is shown, and (b) of the figure schematically shows how the fisheye lens image img to be displayed is mapped onto the plane of the image frame frm, and (c) of the figure. (B) in the figure from the y-axis direction
FIG. Fisheye lens image img
Is an image of the orthographic projection system, the fisheye lens image img of FIG. 2A is an image in which the sphere of FIG. 2B is viewed from the z-axis direction.

Assuming that the fish-eye lens camera is placed upward (in the z-axis direction) at the origin of the x-axis, y-axis, and z-axis, an image projected by the fish-eye lens camera first has a radius of a distance R around the origin. The analysis can be performed assuming a state in which an image is pasted inside the sphere.

Thus, the distance to the object is ignored,
The image img obtained by the fisheye lens camera is such that an image incident from a specific direction (r, θ, φ) is represented by a point (x ″,
y ″). In equation (2), θ is the azimuth, φ is the vertex angle (zenith angle), and L is the image incident at the vertex angle φ on the fisheye lens image. This is the distance (image height) from the origin O when an image is formed.

[0054]

(Equation 2)

That is, the fish-eye lens image img is represented by a two-dimensional (x ″, y ″) from the point of the three-dimensional polar coordinates (r, θ, φ).
Can be thought of as a mapping to The distance (image height) L differs depending on the projection method of the fish-eye lens. In the case of the above-mentioned orthographic projection method, the formula (3-1) is used.
2) In the case of equidistant projection, it is expressed as in equation (3-3). Note that f is the focal length of the lens.

[0056]

(Equation 3)

In FIG. 2B, assuming that the azimuth angle of the image frame frm to be displayed after distortion correction is θ and the vertex angle is φ, panning / tilting operation of the display image is performed by the virtual image frame frm. Is displayed when the image is rotated in the three-dimensional coordinates.

That is, the fisheye lens image img corresponding to the coordinates of each point arranged on the raster within the image frame frm.
By performing the correction process of converting the fisheye lens image img to the original image by specifying the coordinates of the point of the coordinate system and associating the color information signal of the coordinate point with the color information signal of the coordinate point of the display frame frm. Can be. When the camera is facing directly above or below, the pan operation of the image frame frm changes the azimuth angle θ by a predetermined amount, and the tilt operation changes the vertex angle φ by a predetermined amount, and the corresponding fisheye lens What is necessary is just to map the color information signal of the two-dimensional coordinates in the image img.

FIG. 3 is an explanatory view of a photographed image of the fisheye lens camera directed in the horizontal direction. The figure shows a case where the fisheye lens camera is directed in the x-axis direction. In this case, the fisheye lens image img is formed on a vertical plane.

Even in this case, an image without distortion can be displayed by the above-described distortion correction. However, if a fisheye lens image is converted as a video immediately above, a pan / tilt operation is performed. The rotation direction is different from the actual pan / tilt direction of the image. That is, in the case of the image immediately above, the tilt operation was performed by changing the angle φ with the z axis, but when the fisheye lens camera is facing in the horizontal direction, the tilt operation is performed even if the angle φ is changed. No.

Therefore, first, before performing the distortion correction, the direction of the fisheye lens camera, that is, the camera installation angle is corrected. This correction processing is performed by the camera installation angle correction unit 1-.
41.

FIG. 4 is an explanatory diagram for correcting a fisheye lens camera image oriented in the horizontal direction. When the fish-eye lens camera is oriented in the direction of the x-axis of the horizontal axis as shown in FIG. 3, the coordinate system is rotated 90 degrees in advance around the y-axis, and the position of the fish-eye lens image plane img is perpendicular to the z-axis. By rotating the pan / tilt operation, it is possible to perform the same processing as in the case of the image immediately above, for the pan / tilt operation. That is, a tilt operation can be performed by changing the angle φ, and a pan operation can be performed by changing the angle θ.

The example shown in FIG. 4 is a case where the fisheye lens camera is oriented in the horizontal direction. However, when the fisheye lens camera is oriented in any of the upward, downward, horizontal and diagonal directions, the vertical direction is z. The coordinate system for the 'axis is (x', y ', z'), and the coordinates of points on the image frame frm are (x ', y',
z ′), and the point (x ′,
y ′, z ′) and the direction of the fisheye lens camera (z-axis direction) is φ, and a point (x ′, y ′,
Assuming that the angle between z ′) and the x-axis is θ and the angle between the point (x ′, y ′, z ′) on the image frame frm and the y-axis is ρ, the rotation matrix equation of equation (4) is used. By performing the rotation coordinate conversion, the operation direction of pan / tilt can be made to coincide with the actual image. Here, the matrices Rx, Rz, and Ry of the rotation matrix expression of Expression (4) are matrices that provide rotation around the x-axis, y-axis, and z-axis, respectively, and the matrix Rx provides rotation corresponding to the camera angle. .

[0064]

(Equation 4)

For the same azimuth and camera angle (vertex angle), once the equation of the rotation matrix of the above equation (4) is calculated, the coordinates after rotation are obtained as (x, y, z). Next, the mapping of this point to the point (x ″, y ″) of the fisheye lens image on the two-dimensional coordinates can be made to correspond to the point on the image frame and the point of the fisheye lens image. The increase in the amount of calculation for the installation angle of the fisheye lens camera is a slight increase due to the generation of the rotation matrix only once.

The mapping from the point (x, y, z) of the three-dimensional coordinates to the point P (x ″, y ″) of the fisheye lens image on the two-dimensional coordinates is performed by first converting the point (x, y, z) into polar coordinates. The point of expression (r,
θ, φ). This conversion can be performed by the polar coordinate conversion expression of Expression (5).

[0067]

(Equation 5)

Further, in the case of a fisheye lens of the equal area mapping method, the expression is expressed by the expression (3-2). By substituting this expression into the expression (2), the expression of the expression (6) is obtained. Can be x ² + y ² + z ² = x ′ ² + y ′ ² + z ′ ² = r ²
Equation (5) and the basic equation of the trigonometric function are used to simplify the equation (6) to the equation (7).

[0069]

(Equation 6)

[0070]

(Equation 7)

As described above, the correction of the fisheye lens camera installation angle is performed by first converting the coordinates (x ', y', z ') of the rectangular area of the image frame frm by rotating coordinate conversion before performing the image deformation processing. The coordinates of (x, y, z) are converted, and then the points of the (x, y, z) coordinates are converted to points (x ″, y ″) on the coordinates of the fisheye lens image. Since the z-axis component corresponds to the zoom ratio, it is basically a two-dimensional two-dimensional conversion from (x, y) to (x ″, y ″). Equation (7)
In the coordinate conversion equation relating to the equation (1), the number of calculations is reduced by calculating the coordinate conversion associated with the rotation of the coordinate axis and the mapping conversion associated with the characteristics of the fisheye lens at once.

Here, the point (x ″, y ″) on the coordinates of the fisheye lens image obtained by the above calculation is described as (u, v). In order to smoothly interpolate this coordinate point (u, v),
The following interpolation processing is performed. The color information signal components of the three colors of the video signal are Y, U, V, those components of the point (u, v) are Yu, v, Uu, v, Vu, v, and the point (u, v). ), U0 and v0 are values obtained by truncating the decimal points of the coordinate components u and v, and the value of the color information signal Y at the point of the coordinates (u0, v0) is Yu0v0. Similarly,
Regarding the color information signals U and V, similar contents are represented by Uu0v0 and Vu0v0. Further, the values obtained by rounding up the decimal points of the coordinate components u and v are defined as u1 and v1, and the values of the color information signals Y, U and V of the corresponding coordinates are defined as Y and V1, respectively.
They are represented as u1v1, Uu1v1, and Vu1v1. The decimal parts of the coordinate components u and v are du and dv, respectively.

A smooth image can be displayed by interpolating and mapping the input video signal using the primary conversion equation (8).

[0074]

(Equation 8)

Next, a method of automatically extracting a value f corresponding to the image center position (Xbase, Ybase) and the focal length, which are parameters of the fisheye lens camera, from the imaging area of the fisheye lens camera will be described. The above parameters are parameters used in distortion correction of a fisheye lens camera image.

FIG. 5 is an explanatory diagram of parameter extraction of the fisheye lens camera. In the figure, 5-1 is an imaging area of a fish-eye lens camera, and 5-2 is a fish-eye lens image area. Imaging area 5-1 of camera using circular image fisheye lens
Inside, a circular fisheye lens image area 5-2 is observed. The fish-eye lens image area 5-2 is observed at a predetermined position in the camera imaging area 5-1. The outer periphery of the fish-eye lens image area 5-2 is an area having extremely low luminance because light does not reach the area.

Therefore, for the entire camera imaging area 5-1, histograms can be extracted in the horizontal and vertical directions, and the position of the fisheye lens image area and its radius can be measured.

A suitable threshold value T for the horizontal histogram Hh (n) and the vertical histogram Hv (n)
h and Tv are set, the histogram is measured from both ends of the imaging area screen, and a point that exceeds the thresholds Tv and Th is searched first.

Two points are extracted for each of the horizontal and vertical points, and the left end Xleft and right end Xright points of the fisheye lens image area can be detected from the horizontal histogram. Further, it is possible to detect the upper end Yup and the lower end Ydown of the fisheye lens image area from the vertical histogram.

The left end Xref of the fisheye lens image area
From the respective positions of t, right end Xright, Yup, and lower end Ydown, a horizontal radius Rh and a vertical radius Rv can be extracted by Expression (9). or,
The image center position (Xbase, Ybase) is given by equation (10).
Can be extracted by

[0081]

(Equation 9)

[0082]

(Equation 10)

The fisheye lens image area 5-2 is basically circular, and has a horizontal radius Rh and a vertical radius Rv.
And Rh = Rv, but there may be a difference between the horizontal radius Rh and the vertical radius Rv due to the difference in the aspect ratio of the image sensor such as a CCD in the vertical and horizontal directions.

The radius R of the fisheye lens image area 5-2 is
Originally, a value determined by the focal length f of the fisheye lens. For example, it is expressed by Expression (11-1) for an orthographic fisheye lens, and expressed by Expression (11-2) for an equal area projection fisheye lens.
It is represented by

[0085]

[Equation 11]

Therefore, in the calculation of the expression (6) for obtaining the point (x ″, y ″) on the coordinates of the fisheye lens image, a difference occurs between the horizontal radius Rh and the vertical radius Rv. Without using the same value as the value of the focal length f, the value of the horizontal direction radius Rh or the vertical direction radius Rv can be calculated for each direction by the formula (11-1) or the formula (11-2).
By correcting the value of f by substituting it into the value of R, the distortion of the fisheye lens can be corrected in consideration of the difference in the aspect ratio.

FIG. 6 is a diagram showing the configuration of the moving image detecting device according to the first embodiment of the present invention. In the figure, 6-1
Denotes a fisheye lens camera, 6-2 and 6-3 denote frame buffers, 6-4 and 6-5 denote inter-frame difference calculation units, 6-6 denotes an area weight table, and 6-7 denotes an area extraction unit.

The image signal photographed by the fish-eye lens camera 6-1 is stored in the first frame buffer 6-2.
The image signal stored in the frame buffer 6-2 is stored in the second frame buffer 6-3. The second frame buffer 6-3 stores the image signal of the previous frame.

Then, the image signal output from the first frame buffer 6-2 and the second frame buffer 6-3
And the first frame difference calculator 6-4 calculates the difference between the frames based on the image signal output from the second frame.
Is output to the inter-frame difference calculator 6-5.

The second inter-frame difference calculator 6-5 corresponds to the inter-frame difference signal output from the first inter-frame difference calculator 6-4 and the area output from the area weight table 6-6. The inter-frame difference signal is weighted in accordance with the region by the obtained value, and the signal is divided into the region extraction unit 6-6.
7 is output.

The area extracting section 6-7 detects a moving image based on an inter-frame difference signal having a weight corresponding to the area, outputs a detection flag signal, extracts the detected area, and outputs area information. .

FIG. 7 is a diagram showing a configuration of a moving image detecting device according to the second embodiment of the present invention. In the figure, 7-1
Is a fisheye lens camera, 7-2 and 7-3 are frame buffers, 7-4 is an inter-frame difference calculator, 7-5 is an intra-frame difference calculator, 7-6 is a moving contour extractor, and 7-7 is a region weight. A table, 7-8 is an area extraction determination unit, and 7-9 is an area extraction unit.

The image signal photographed by the fish-eye lens camera 7-1 is stored in the first frame buffer 7-2, and is stored in the first frame buffer 7-2.
The image signal stored in the frame buffer 7-2 is stored in the second frame buffer 7-3. The second frame buffer 7-3 stores the image signal of the previous frame.

Then, the image signal output from the first frame buffer 7-2 and the second frame buffer 7-3
, An inter-frame difference is calculated by an inter-frame difference calculation unit 7-4, and an inter-frame difference signal having a large value with respect to the moving image is output to a moving contour extraction unit 7-6. .

The intra-frame difference calculation section 7-5 calculates the intra-frame difference of the image signal output from the first frame buffer 7-2, and outputs the intra-frame difference signal having a large value at the outline of the image. Output to the moving contour extraction unit 7-6.

The moving contour extraction unit 7-6 linearly combines the inter-frame difference signal from the inter-frame difference calculation unit 7-4 with the intra-frame difference signal from the intra-frame difference calculation unit 7-5, Generates a signal of a moving image in which is emphasized, and outputs the signal to the region extraction determination unit 7-8.

The region extraction judging section 7-8 includes the moving contour extracting section 7
The signal of the moving image is weighted according to the region by the signal of the moving image in which the contour portion is output from -6 and the value corresponding to the region output from the region weight table 7-7. The signal is output to the region extraction unit 7-9.

The area extracting section 7-9 detects a moving image based on a moving image signal having a weight corresponding to the area, outputs a detection flag signal, extracts the detected area, and outputs area information. .

The moving image detecting apparatus according to the embodiment of the present invention shown in FIGS. 6 and 7 weights a video signal corresponding to a region in extracting a region of a moving image, and detects a detection parameter based on image distortion by a fisheye lens. The effect on the signal is reduced.

That is, normally, the video signal P of n frames
_n (x, y) and the video signal P _n-1 (x,
When calculating the inter-frame difference D _n (x, y) with respect to y), the calculation is performed by equation (12). In the present invention, the weight coefficient W (x, x) of the feature extraction corresponding to the area of the fisheye lens image is calculated.
y) are stored in the area weight tables 6-6 and 7-7, and the weight coefficient W (x, y) is calculated by using the inter-frame difference D
_A moving image is extracted using a value G _n (x, y) multiplied by _n (x, y). _{D n (x, y) =} P n (x, y) -P n-1 (x, y) ··· (12)

That is, the first inter-frame difference shown in FIG.
An inter-frame difference signal output from the calculating unit 6-4, or
The frame output from the moving contour extraction unit 7-6 shown in FIG.
The signal obtained by linearly combining the inter-system difference and the intra-frame difference is
Feature value D for moving image extraction _n(X, y),
According to equation (13), this feature amount D_n(X, y) and fisheye
G with weighting coefficient W (x, y) corresponding to the region of the lens image
_n(X, y) is used as a feature for extracting a moving image.
You. G_n(X, y) = W (x, y) · D_n(X, y) (13)

As described above, by performing the extraction using the characteristic amount Gn (x, y) weighted according to the area of the fisheye lens image, an equivalent mask and enhancement operation corresponding to each area in the fisheye lens image area are performed. It can be performed.

FIG. 8 is an explanatory diagram of the weight coefficient W (x, y) according to the area of the fisheye lens image. In the figure, the inside of the area of the fisheye lens image is divided into three areas 8-1, 8-2, and 8-3. The area 8-1 has a weight coefficient of 0, the area 8-2 has a weight coefficient of 1, and the area 8 -3 shows an example in which a weighting factor of 2 is assigned.

In the peripheral portion of the fisheye lens image, a region having the same area as the central portion is displayed in a smaller region. Therefore, the accuracy of detecting the moving image in the peripheral portion is reduced. However, by setting the weighting coefficient W (x, y) of the peripheral portion to a large value, it is possible to compensate for the decrease in accuracy in the peripheral portion. Further, since there is usually a high probability that a human image exists in the peripheral portion, the peripheral portion is set to a large value and the central portion is set to a value close to 0.

Further, since the imaging area of the fisheye lens camera is very large, illumination and the like often directly enter the imaging area. Even in such a case, an extreme difference in luminance can be compensated by an equivalent area mask to reduce erroneous detection of a moving image.

The region extraction units 6-7 and 7-9 convert the feature values weighted corresponding to the regions into predetermined threshold values (T
hm), an image detection determination process for a human image or the like is performed. The region extraction processing accompanying the image detection determination processing is performed for the entire screen of the fisheye lens image.

FIG. 9 is a diagram showing an example of an image of a person or the like whose area has been extracted. (A) of the figure shows a fisheye lens image area, and shows an example of a fisheye lens image when a fisheye lens camera is placed face up on a desk or the like, and four persons are seated around the camera. Reference numerals 9-1 to 9-4 denote person image detection areas (A) to (D). (B) of the figure shows the shape of the human image of the fish-eye lens image. When a person is photographed with a fish-eye lens from below, the body has a long trapezoidal shape with a long bottom and a short head.
The human image can be normalized to the shape.

When a point having a large feature amount as a human image is detected by raster-scanning the image of the fish-eye lens image area as shown in FIG. 9A, an image having a substantially trapezoidal shape is drawn downward from the point. Search and cut out the area.

The size and shape of the cut-out area are collated to determine whether or not the cut-out area is a person image area. The same process is simultaneously performed on a plurality of regions, and when a plurality of human images are detected, a plurality of human images are simultaneously cut out.

For each cut-out area, the parameters of the top, bottom, and height of the person and color data inside the person as shown in FIG. 9B are sampled in the left, right, and height directions. This can be used as a database for identifying a human image for each image.

As for the process of extracting a region such as a human image from the extracted feature amount, an algorithm for region extraction in a normal camera image can be basically used, but the fisheye lens image is distorted in a circular shape. It is necessary to perform coordinate conversion because they are arranged in the same manner.

There are the following two methods for coordinate conversion. The first method is a method of performing mapping conversion of a fisheye lens image into a coordinate-converted image as a 360-degree panoramic image. This method converts a circular fisheye lens imaging area into a rectangular area.

The second method is a method in which the area is scanned as a normal raster scan, and only the coordinate system at the time of scanning is converted from rectangular coordinates (x, y) to polar coordinates (r, θ). That is, region extraction processing is performed with x = rcos θ and y = rsin θ.

The first method has a drawback in that a large memory is required for developing a panoramic image, but after conversion, a territory extraction process for a normal image can be applied as it is. In the second method, processing must be performed while always performing coordinate conversion at the time of region extraction processing. However, the memory amount to be used is only the memory amount for providing a fisheye lens image as an original image, and a compact system can be constructed.

Then, the feature amount is extracted, and the integration processing of the regions is performed by utilizing the mutual relation of the known images such as the vertical relation.
For example, in the case of a fish-eye lens camera that faces directly upward, regions are integrated using the positional relationship of the video with the center of the fish-eye lens image up and the periphery down, and the region is changed from a background image to a human image Is extracted.

With respect to tracking processing of a moving image such as intrusion of a person, a grid point that equally divides the cut-out substantially trapezoidal human area vertically and horizontally by a predetermined number of divisions is calculated. Is stored as a feature amount of the human image. Then, the registration processing of the feature amount of the human image and the extraction processing of the human image are repeatedly performed at an arbitrary time interval, and even when a plurality of human images are present in the fisheye lens camera imaging area, each human image is individually tracked. can do.

FIG. 10 is a flowchart of a process for determining a person image extraction position according to the embodiment of the present invention. First, a command for cutting out a human image or the like in a fisheye lens image is given (10-1), and a loop process (10-
According to 2), a feature amount of a human image or the like based on the inter-frame difference or the inter-frame difference and the intra-frame difference is extracted (10-3), and a weighting process corresponding to a region in the fisheye lens image is performed (10-4). Then, a raster loop process in the fisheye lens screen is performed (10-5), a feature point is detected (10-6), and a coordinate conversion of a fisheye lens image is performed for an area on a substantially trapezoid to extract a feature point ( 10
-7), it is determined whether there is a feature point (10-8),
If there is a feature point, the extraction point is deleted (10-9),
The maximum value and the minimum value of the (x, y) coordinate area are stored (1
0-10), and determines whether or not it is the last line (10-1).
1) If it is the last line, the process ends (10-12).

It is determined whether or not the feature point exists (10-
In 8), if there is no feature point, it is determined whether there is no feature point for a plurality of continuous lines (10-13). If there is no continuous feature point, the number of areas is counted and the person area is stored. (10-14), it is determined whether or not it is the last line described above, and the routine goes to (10-11).

In the determination (10-13) as to whether or not there is a feature point in a plurality of continuous lines, if the plurality of lines are not continuous, the process returns to the process of detecting the feature point (10-6). Also, it is determined whether the line is the last line (10-1).
If it is not the last line in 1), the process returns to the process (10-6) for detecting the feature point.

When the region where the human image exists is extracted, parameters for correcting and displaying the fisheye lens image can be obtained from the region. FIG. 11 is an explanatory diagram of a configuration for extracting an area and creating a display screen according to the embodiment of this invention. In the figure, 11-1 is an area extraction unit, 11-2
Denotes an area cutout unit, 11-3 denotes a fisheye lens image parameter determination unit, and 11-4 denotes a display screen creation unit.

The area extracting section 11-1 is the same as the area extracting sections 6-7 and 7-9 shown in FIGS. 6 and 7 and detects a person image or the like based on the feature amount. Is extracted, a signal of a detection flag and area information is output. The signal of the detection flag can be used as an alarm or notification signal. The area information signal is output from the area cutout unit 11-2.
Is output to

The area cutout section 11-2 cuts out an area based on the signal of the area information, and the fisheye lens image parameter determination section 11-3 sets parameters for correcting and displaying the fisheye lens image for all the cutout areas. It is determined, and the signal of the parameter is sent to the screen creation unit 11-4. The screen creating unit 11-4 generates a display screen based on the parameter signal and outputs a video signal.

Here, for the sake of simplicity, the determination of the parameters of the fisheye lens image in the case of a camera angle pointing directly upward will be described. As the parameters for correcting the fisheye lens image, the azimuth angle θ, the vertex angle φ, and the enlargement factor m of the image frame are required.

From the area of the fisheye lens image, for example, FIG.
(A) of the human image detection area (B) 9-2 is extracted and displayed. In this case, the person image detection area (B) 9-
The angle (solid angle) β that sandwiches the human image detection area (B) 9-2 from the angle α related to the center position 2 and the center O of the fisheye lens image is calculated. Further, a distance L from the center O of the fisheye lens image to the top of the area is calculated.

The azimuth angle θ of the image frame displaying the human image detection area (B) 9-2 is determined by the angle α and the distance L.
And the value of the vertex angle φ can be obtained. That is, the azimuth θ
Is equal to the angle α, and θ = α. Also, the vertex angle φ
Can be obtained from the ratio between the radius R of the fisheye lens image and the distance L by using Expressions (3-1) to (3-3). The zoom rate m can be obtained from the solid angle β between the left end and the right end of the area and the center O of the fisheye lens image. Based on these parameters, the display area of each person is cut out, the distortion of the fisheye lens image is corrected, and a video signal is transmitted to a monitor television or the like.

FIG. 12 is a diagram showing an image obtained by synthesizing images taken by one fish-eye lens camera. As described above, a human image is extracted from the fisheye lens image by its feature amount,
FIG. 3 schematically shows a state in which a distortion correction process is performed on an extracted image, and one monitor television screen is divided into four to display the extracted human image.

[0127]

As described above, according to the present invention,
Since images are captured by a fish-eye lens camera with a wide angle of view of about 180 degrees, a single camera can monitor a wide area and reduce the number of installed cameras, and can accurately and quickly capture the images. It can be corrected and displayed.

In addition, by performing a combination of the coordinate conversion for correcting the installation angle of the fisheye lens camera and the coordinate conversion for correcting the distortion of the fisheye lens image, the image correction calculation process can be performed at high speed regardless of the installation angle. Can be. or,
By extracting parameters for fisheye lens image distortion correction from the captured fisheye lens image itself and correcting the distortion of the fisheye lens image, stable distortion correction can be performed for adjustment during installation and aging. Can be.

By extracting a region based on a feature amount obtained by multiplying different weighting factors in accordance with the magnitude of the distortion between the central portion and the peripheral portion of the fisheye lens image, a human image or the like in the peripheral portion having a large distortion can be obtained. Detection can be performed in a stable manner. Furthermore, by assigning different values as weighting factors corresponding to regions in the image of the fisheye lens image and masking the non-extracted regions, the non-extracted region in the extraction process of a human image or the like can be prevented. The stabilizing element can be removed in advance.

Further, the present invention has a high degree of freedom in creating an image, for example, by cutting out an arbitrary portion from a wide-range image captured by a fish-eye lens camera and displaying it on a monitor television screen. Thereby, convenience can be improved.

Further, an image accompanying movement such as intrusion of a person is detected from the video projected by the fish-eye lens camera, and a monitor display area (cutout area) of the image is automatically determined to perform smooth movement. An electronic tracking type remote monitoring system or a video conference system for tracking can be configured.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a fisheye lens camera device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an image captured by a fisheye lens camera according to an embodiment of the present invention.

FIG. 3 is an explanatory diagram of a photographed image of a fisheye lens camera oriented in a horizontal direction.

FIG. 4 is an explanatory diagram of correction of a fisheye lens camera image oriented in a horizontal direction.

FIG. 5 is an explanatory diagram of parameter extraction of a fish-eye lens camera.

FIG. 6 is a diagram illustrating a configuration of a moving image detection device according to the first embodiment of the present invention.

FIG. 7 is a diagram illustrating a configuration of a moving image detection device according to a second embodiment of the present invention.

FIG. 8 is a diagram illustrating a weight coefficient W according to a region of a fisheye lens image.
It is explanatory drawing of (x, y).

FIG. 9 is a diagram showing an example of an image of a person or the like from which an area has been extracted.

FIG. 10 is a flowchart of a process of determining a human image extraction position according to the embodiment of the present invention.

FIG. 11 is an explanatory diagram of a configuration for cutting out an area and creating a display screen according to the embodiment of this invention.

FIG. 12 is a diagram illustrating an image obtained by synthesizing images captured by one fish-eye lens camera.

FIG. 13 is a diagram showing a configuration of a conventional fish-eye lens camera device, and a remote monitoring system and a video conference system using the same.

FIG. 14 is an explanatory diagram of distortion correction of a conventional fisheye lens camera image.

[Explanation of symbols]

 1-1 Fisheye lens 1-2 CCD imaging device 1-3 Video memory (frame memory) 1-4 Image correction processing unit 1-41 Camera installation angle correction unit 1-42 Pan / tilt angle rotation correction unit 1-43 Zoom correction unit 1-44 Video Interpolation Creation Unit 1-5 Corrected Video (NTSC) Output Circuit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FI H04N 5/781 520A

Claims (18)

[Claims] 1. A fisheye lens camera device comprising an image correction processing unit for correcting distortion of an image photographed by a fisheye lens camera, wherein the image correction processing unit includes a coordinate transformation for correcting an installation angle of the fisheye lens camera, and a fisheye lens image. A fisheye lens camera device having a configuration for performing an operation in combination with a coordinate transformation for correcting a distortion of the fisheye lens. 2. The fisheye lens camera device according to claim 1, wherein said image correction processing unit has a configuration for calculating coordinate transformation for correcting distortion of a fisheye lens image of an equal area mapping. 3. A fisheye lens camera device comprising an image correction processing unit for correcting distortion of an image taken by a fisheye lens camera, wherein a center position of a display image, an aspect ratio of the fisheye lens image, A fish-eye lens camera device comprising: a configuration in which parameters such as a radius of a fish-eye lens image area are extracted from a captured fish-eye lens image itself, and distortion correction of the fish-eye lens image is performed based on the extracted parameters. 4. A fish-eye lens camera apparatus for extracting a moving image from an image taken by a fish-eye lens camera, wherein regions are integrated by utilizing mutual positional relationships in the fish-eye lens image, and a region in which the image has changed is extracted. A fish-eye lens camera device comprising: 5. A fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, wherein the fish-eye lens image is converted into a panoramic landscape image, and a signal of an inter-frame difference of the image or an inter-frame difference and a frame A fish-eye lens camera device, comprising: a signal for extracting a signal obtained by combining an internal difference, extracting a region based on the signal, and simultaneously extracting regions of a plurality of moving images. 6. A fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, wherein a calculation of a feature amount such as a signal of an inter-frame difference or a signal combining an inter-frame difference and an intra-frame difference is performed. Based on the image of the fisheye lens image, in the area extraction process for the feature detection area, the address conversion between polar coordinates and rectangular coordinates is performed, the inside of the image of the fisheye lens image is scanned, and a plurality of circular fisheye lens image images are scanned. A fish-eye lens camera device comprising a configuration for extracting a region of a moving image. 7. A fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, wherein the shape of each human image is normalized with respect to a plurality of human image regions obtained from the fish-eye lens image. A fish-eye lens camera device comprising a configuration for recording color information in the shape of each image and creating an image for individually tracking a plurality of moving human images based on the color information of each image. . 8. A fish-eye lens camera device for extracting a moving image from an image taken by a fish-eye lens camera, wherein a feature amount obtained by multiplying different weight coefficients according to the magnitude of distortion between a central portion and a peripheral portion of the fish-eye lens image. A fish-eye lens camera device having a configuration for performing region extraction on the basis thereof. 9. The apparatus according to claim 8, wherein a different value is further provided as said weighting factor in accordance with a region in the image of the fisheye lens image, and a non-extracted region is masked to perform region extraction. The fisheye lens camera device as described in the above. 10. A method of correcting distortion of an image photographed by a fish-eye lens camera, comprising a step of performing a combination of coordinate transformation for correcting an installation angle of the fish-eye lens camera and coordinate conversion for correcting distortion of a fish-eye lens image. Characteristic image distortion correction method for fisheye lens camera. 11. The method for correcting image distortion of a fish-eye lens camera according to claim 10, wherein the correction of the image distortion of the fish-eye lens camera includes a step of calculating a coordinate transformation for correcting the distortion of the fish-eye lens image of the equal area mapping. . 12. A method for correcting distortion of an image captured by a fish-eye lens camera, comprising: capturing parameters such as a center position of a display image, an aspect ratio of the fish-eye lens image, and a radius of a fish-eye lens image area for correcting distortion of the fish-eye lens image. An image distortion correction method for a fish-eye lens camera, the method including a step of extracting from the obtained fish-eye lens image itself and correcting the distortion of the fish-eye lens image using the parameters. 13. An image extracting method for extracting a moving image from an image taken by a fish-eye lens camera, wherein the regions are integrated using mutual positional relationships in the fish-eye lens image, and a region in which the image is changed is extracted. An image extraction method for a fish-eye lens camera, comprising: 14. An image extraction method for extracting a moving image from an image taken by a fish-eye lens camera, comprising: converting a fish-eye lens image into a panoramic landscape image; An image extraction method for a fish-eye lens camera, comprising: extracting a signal obtained by combining an internal difference, extracting an area using the signal, and simultaneously extracting areas of a plurality of moving images. 15. An image extraction method for extracting a moving image from an image captured by a fish-eye lens camera, the method comprising: extracting a feature amount such as a signal of an inter-frame difference or a signal combining an inter-frame difference and an intra-frame difference. Based on the image of the fisheye lens image, in the area extraction process for the feature detection area, the address conversion between polar coordinates and rectangular coordinates is performed, the inside of the image of the fisheye lens image is scanned, and a plurality of circular fisheye lens image images are scanned. An image extraction method for a fish-eye lens camera, comprising a step of extracting a region of a moving image. 16. An image extraction method for extracting a moving image from an image captured by a fish-eye lens camera, wherein the shape of each human image is normalized for a plurality of human image regions obtained from the fish-eye lens image video, Image of a fish-eye lens camera, including a process of recording color information in the shape of each image and creating an image for individually tracking a plurality of moving human images based on the color information of each image Extraction method. 17. An image extraction method for extracting a moving image from an image captured by a fish-eye lens camera, wherein a feature amount obtained by multiplying different weighting factors according to the magnitude of distortion between a central portion and a peripheral portion of the fish-eye lens image. An image extraction method for a fish-eye lens camera, comprising a step of extracting a region based on the image. 18. The method according to claim 17, further comprising the step of giving different values corresponding to regions in the image of the fish-eye lens image as the weighting factors and masking non-extracted regions to perform region extraction. Fisheye lens camera image extraction method.