JPH0515111B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention relates to an image reproducing apparatus that corrects lens distortion of an electronic still camera by signal processing during image reproduction. In a patent application titled "Electronic Still Camera" dated the same date as the present application, the present inventors converted two-dimensional image information from a subject into an image signal using an image sensor, A-D conversion of this image signal, and conversion. We have proposed an electronic still camera that stores digital image information in a removable storage means. Since this electronic still camera digitally processes image signals, it can obtain high-quality still images without signal deterioration during signal processing. Also, since digital image data is stored in memory, there is no need for a rotating mechanism, reducing power consumption. It is superior to conventional electronic still cameras in many ways, including a smaller camera body and improved reliability. Figure 1 shows the basic circuit configuration of this electronic still camera, where 1 is an optical lens that forms an optical image of a subject, and 2 is an optical lens 1.
An image sensor that photoelectrically converts an optical image (two-dimensional image information) formed on a light-receiving surface and outputs it as an analog image signal.Generally known types include Sachicon, Vidicon, Plumbicon, Cosbicon, and Newcos Bicon. image pickup tubes and CCDs,
There are solid-state image sensors such as MOS and CPD. A color filter 2a is attached to the light receiving surface of the image sensor 2 so that a color image can be obtained. The image sensor 2 outputs an analog image signal in response to a drive signal from a driver 3, and this image signal is converted into a digital signal at high speed by an A/D converter 4.
In this case, the quantization level is determined by the required image quality, but approximately 8 bits are required for general television images. Generally speaking, a sampling rate that is 2 to 4 times that of the color subcarrier (3.58MHz) is required, but in electronic still cameras, this is not necessarily the case if the imaging unit has a temporary storage function. Not. The image signal digitally converted by the A/D converter 4 is stored in the memory 5. As this memory 5, anything can be used as long as it can store digital signals stably, such as N-MOS RAM, C-MOS RAM,
In addition to semiconductor memories such as MOS RAM and CCD memory, magnetic bubble elements, magnetic disks, and magnetic tapes are used. Writing and reading of image signals to and from the memory 5 is controlled by a driver 6. The memory 5 is of a replaceable type, for example, in the form of a cassette, which can be attached to and removed from the camera body at a predetermined position from the outside. Depending on the type of memory, batteries may also need to be included in the cassette. On the other hand, the digital image signal output from the A/D converter 4 is directly or once stored in the memory 5 and then converted into an analog image signal by the D/A converter 7. The image is then sent to the computer and displayed as an image on a monitor CRT. In this case, the contacts a and b of the changeover switch 8 are changed over by a changeover signal from the driver 6 by external operation. Further, the output of the image sensor 2 can also be visualized by an electronic viewfinder 9 via a direct changeover switch 8 (contact a). In this case, since the output of the image sensor 2 is modulated by the color filter 2a, when the image is displayed directly on the electronic viewfinder 9, it is different from a normal black and white image, but it is possible to know what is being displayed. . Note that 10 is a battery that serves as a power source for each of the circuit elements described above. FIG. 2 is a block diagram of one embodiment of the player, in which the memory 5 is removed from the camera and set in a predetermined position in the player (shown surrounded by a broken line). In the playback machine, the microcomputer 12 is the storage means 13.
The video data recorded in the memory 5 is read out according to the software in the ROM 131 inside the RAM
132 is used as a work area for data processing.
The processed data is written to the field memory 14 by commands from the microcomputer 12, but this field memory 14 is always driven by the TV rate drive circuit 15 except when accessed by the microcomputer 12. The output is displayed as an image on the monitor 17 via the D/A converter 16. As a result, the recorded video gradually appears on the monitor 17. In the electronic still camera described above, an optical image of the subject is formed by an optical lens on the light receiving surface of an image sensor, photoelectrically converted, and output as an analog image signal.
When using a solid-state image sensor such as a MOS, there is a phenomenon in which the image point is distorted outward or inward toward the periphery of the lens due to lens distortion. Third
The figure shows this situation. If the circles in the figure are the light receiving positions of each pixel of the solid-state image sensor, the straight line that should ideally be imaged as the thick line becomes the broken line due to lens distortion. The image will be distorted like this.
For this reason, for example, an image that would normally be formed at the light receiving position of C5 is now formed at the light receiving position of B5, and if the image signal obtained in this way is reproduced as it is, the original linear lattice pattern will be distorted. The image is projected as a distorted grid pattern shown by broken lines. The degree of this distortion varies depending on the lens. In order to eliminate such lens distortion, conventional efforts have been made at the lens manufacturing stage to adjust the diameter and thickness of the lens or the curvature of the lens surface, or to increase the number of combined lenses. There are problems in that the lens diameter becomes large or heavy, and the cost becomes high. For this reason, the reality is that the lenses of recent video cameras and electronic still cameras are much larger than the main bodies. The present invention corrects lens distortion in an electronic still camera by electrically processing image signals without using any ingenuity in lens manufacturing. The image signal is stored every time the image signal is stored, and the read address of the image signal is corrected based on the distortion information when the image is reproduced. The present invention will be explained in detail below based on the drawings. The basic concept of the lens distortion correction method according to the present invention will be explained using FIG. As a diagram corresponding to Figure 3, Figure 4 shows the light receiving position of each pixel of the solid-state image sensor (9 x 9 pixels) with circles, and the linear grid (solid line) that is the subject.
shows a state in which the image is formed as a pincushion-like lattice (dashed line) due to lens distortion. At this time, image signals are stored in an arrangement as shown in Table 1 below, corresponding to each pixel in the image signal memory.

[Table] When reproducing such a memory, in order to correct lens distortion, in the case of pincushion distortion (Figure 4), each pixel must be rearranged as shown in Table 2 below. Read the image signal from.

[Table] In the case of barrel distortion (Fig. 3), the data should be read out after being rearranged into the arrangement shown in Table 3.

【table】

[Table] Lens distortion can be corrected by reading out as shown in Table 2 or Table 3 above when reproducing image signals, and the reproduced image is displayed as a substantially linear lattice pattern. If the lens distortion is not pincushion-shaped, as shown in Figure 3, image information at the top, bottom, left, and right edges of the screen will be lost, but as is well known, on a television screen, all of the reproduced image signals are lost. Since the images are not displayed within the TV frame, there is no practical problem as long as the image information that is lost falls within this range. If this point is taken into account, the distortion tolerance will be a few percent, so it will be possible to make the lens considerably smaller and lighter. By the way, the following several methods can be considered as specific methods for reading out image signals for correcting lens distortion. (1) Now let the coordinates in the pixel of the solid-state image sensor be (x _o ,
y _o ), and optical information that should originally be incident on that pixel is incident on point Q (x _o ', y _o ') due to lens distortion. In this case, |(x _o ′, y _o ′)−(x _n ,
Select a point R|(x _n , y _n ) such that y n ) | is minimum, and convert the signal of the pixel at this point R to the pixel P ( _{x o} ,
y _o ) is used as a signal. (2) Similar to (1) above, pixel P(x _o ,
Suppose that optical information that should originally be incident on point Q (xo _' , yo') is incident on point Q (xo' _, yo _' ) due to lens distortion. In this case _, the four points (x _n1 , y _n1 ), (x _n2 , y _n2 ) _{, (} x _{n3 ,}
y _n3 ), (x _n4 , y _n4 ), and the signal calculated by appropriately weighting the signal of the light receiving section at each point is set as the pixel signal of point P (x _o , _yo ). Weighting is determined according to the sensitivity shape of the pixel, and if the sensitivity of each pixel decreases linearly in proportion to the distance from its center position regardless of direction, then point Q and the above 4.
Weighting is performed in proportion to the distance to the point, and if the change in sensitivity of each pixel has a special directionality, weighting is performed taking that directionality into consideration. It may be determined based on where the point Q exists in a predetermined area with respect to the point P, regardless of the sensitivity of the weighted pixel. Therefore, in the present invention, lens distortion information, which is information that determines how to read an image signal, is stored for each screen at the time of imaging, but the following storage method can be considered. (1) Utilization of free space in memory: For example, when using an image sensor with 385 horizontal pixels, one scanning line is usually composed of 512 bytes for ease of signal processing, but this results in unnecessary waste. The memory area can be used to store lens distortion information. (2) Use of one scanning line: One screen of image sensor information is scanned with 525 scanning lines, of which the first few and last few scanning lines are used on the monitor screen of the playback device. is not displayed. Therefore, lens distortion information can be stored in an area corresponding to this scanning line portion. (3) Use of horizontal or vertical retrace periods: When using memory with a mechanical drive mechanism such as a magnetic disk or magnetic tape as the memory of an electronic still camera, these memory media must be used even during the retrace period. Since the lens is in operation, lens distortion information can be stored in a portion of the memory medium corresponding to this retrace period. FIG. 5 is a block diagram when a memory is set in a reproducing device (encircled by a broken line) to reproduce an image. The playback device is basically the same as that shown in FIG. 2, and is characterized by having a ROM 18 such as a programmable ROM that stores several different lens distortion processing data 18a, 18b, 18c, . . . . Now, when you set the memory 5 containing images taken with an electronic still camera into the playback device, the memory 5
Since the lens distortion information used in the camera is stored together with the image information, the distortion information is read out by a command from the microcomputer 12 and temporarily stored in the storage means 13 in the playback device.
In addition to writing to RAM132, memory 5
The image information within is transferred to the playback memory 14. At this time, some distortion processing data 18a, which is stored in the ROM 18 and which corresponds to the type of electronic still camera to which the memory 5 set in the playback machine can be attached or detached, that is, according to the type of camera lens, is stored.
Among 18b, 18c..., distortion processing data corresponding to the distortion information read out from the memory 5 currently set in the playback machine and written to the RAM 13 is read out, and the image information successively sent to the playback memory 14 is processed. do. The image information stored in the reproduction memory 14 is read out at, for example, a television rate, converted into an analog image signal by the D/A converter 16, and displayed on the monitor 17. ROM during the above explanation
The ROM may be a normal ROM or a programmable ROM, and the ROM 18 and the like may be configured to include a ROM that can be attached or removed for each data to be processed. As explained above, the present invention directly A-D converts an image signal obtained by making light from a subject enter an image sensor through an optical lens, and converts this converted digital information into a removably installed device. In an electronic still camera, distortion information of the optical lens is stored in the storage means each time an image signal for one screen is stored, and this distortion information is used during image playback. Lens distortion can be corrected. Specifically, the readout address of the image signal stored in the storage means is corrected based on lens distortion information. In this way, there is no need to correct lens distortion at the lens manufacturing stage, so it is possible to miniaturize the lens and reduce costs without increasing the diameter or weight of the lens. I can do it.

[Brief explanation of the drawing]

Figure 1 is a schematic configuration diagram of an electronic still camera, Figure 2
3 is a schematic diagram of the reproducing machine, FIGS. 3 and 4 are explanatory diagrams for explaining the state of lens distortion, and FIG. 5 is a block diagram of the reproducing machine according to the present invention, showing a state in which a memory is set. 1...Optical lens, 2...Image sensor, 4...
A/D converter, 5... Memory, 6... Driver, 7, 16... D/A converter, 8... Switch, 9... Electronic finder, 10... Battery, 12... Microcomputer, 131, 1
8...ROM, 132, 19...RAM, 14...
...Field memory, 15...TV rate drive circuit, 17...Monitor.

Claims (1)

[Claims] 1. In an image reproducing device that reproduces an image signal captured by an electronic still camera, a plurality of distortion processing data for correcting the image signal according to the type of distortion information of the optical lens used at the time of photography. a storage unit that stores the image signal and the distortion information of the optical lens, which is attached to the apparatus main body, and selects one of the distortion processing data corresponding to the distortion information; An image signal reproducing apparatus comprising: a control means for controlling readout of the image signal in the storage means based on distortion processed data.