JPH0432350B2 - - Google Patents

Description

[Detailed Description of the Invention] The present invention relates to a synthetic aperture radar mounted on an artificial satellite or an aircraft.
The present invention relates to a digital device for reproducing human-understandable images from photographic data captured by SAR (hereinafter abbreviated as SAR), and particularly relates to an imaging data processing device suitable for high-speed processing of SAR data.

In the field of moat sensing using artificial satellites or aircraft, SAR is attracting attention as a sensor for photographing the ground surface, as it uses microwave bands that penetrate clouds and can provide high-resolution images. Regarding the SAR itself, for example, “A
Digital Processor for The Production of
Seasat Synthetic Aperture Radar Imagery，
(by JRBennett, IGCumm−ing, July 16−
18, 1979, Italy), so a detailed explanation will be omitted, but the basic idea is that it was published in 1951.
This is said to be the concept of Dotspra sharpening proposed by C. Wiley. The azimuth resolution obtained by measuring Doppler shift can be improved to about 1/2 of the actual aperture diameter of an aircraft antenna, and this can also be explained by the idea of a synthetic aperture antenna. In other words, the actual antenna installed on the aircraft is one element of a virtual one-dimensional array antenna of length L arranged in the direction of flight of the aircraft, and the positions of the elements of the virtual array are sequentially determined. By memorizing the amplitude and phase of the received echo at each point, and later combining them, we can essentially capture the target using an antenna with an aperture of a certain length. is equivalent to .

FIG. 1 shows the overall image processing system according to the present invention. The ground station _Ls receives data from the SAR mounted on an aircraft. R _s is radar sensor,
A _o is the antenna, F _p is the flight path, C is the decomposition cell, R _a is the range direction on the ground surface of the data collected by SAR, A _z is the azimuth direction, A _B is the antenna beam and C _w indicate the cutting width. The data received by the ground station (tracking station) is processed by the data processor D _P.
A video film IF is created, or a magnetic tape MT is created to store processed data.

The problem with ground stations is that they are installed on aircraft, etc.
The problem lies in how to process signals from SAR at high speed to improve so-called throughput.

First, we will provide an overview of SAR processing.

In the received SAR image, one point of the original image is distributed with the spread of a point spread pattern h (x, y), and the received SAR image is not an image that humans can understand as it is. The image becomes comprehensible only when the information spread out in the received image is compressed into a single point through two-dimensional correlation processing of the received image and a point image pattern. This two-dimensional correlation processing can be performed in two stages: correlation processing in the range direction and correlation processing in the azimuth direction. That is, first, received image data in the range direction is extracted line by line and correlation processing with the point image data in the range direction is performed. This process compresses information in the range direction.
When the compression in the range direction is completed, the data is extracted line by line in the azimuth direction, and correlation processing with the azimuth direction point sequence data is performed. When compression in the range and azimuth directions is completed, the SAR received image becomes an image that humans can understand, and then distortion correction is performed as necessary and used in various fields.

Here, when performing SAR image reproduction processing, calculating the correlation processing numerically according to its definition formula is inefficient when the amount of data is huge like SAR, so it is recommended to reduce the amount of calculation and calculation time. Methods using fast Fourier transform (FFT), complex multiplication, and inverse fast Fourier transform (IFFT) are used.

The above is the basics of SAR data processing, but the following three SAR-specific data processings are also required. These are range car drift correction, multi-lux processing, and corner turn processing.

The purpose of range curve correction is to remove blur patterns that appear in processed images due to changes in the distance (range) between the SAR sensor and the imaging target during flight of the satellite or aircraft. For this purpose, in the SAR imaging algorithm, data on a quadratic curve determined by the orbit of a satellite or aircraft is obtained in the frequency space during azimuth direction compression processing (by one-dimensional interpolation processing in the range direction), and a new array is created. It needs to be data.

The purpose of multi-look processing is to reduce speckle noise, which is characteristic of coherent systems such as SAR processing, and to obtain smooth images. For this purpose, in the SAR imaging algorithm, during azimuth compression, the frequency band of the point sequence is divided, a compressed image is obtained for each divided band, and the sum of these images is calculated. Although the resolution of the compressed image for dividing the frequency band may drop to some extent, good image quality can be obtained.

For a corner turn, a range-compressed line piece containing image data for one line in the azimuth direction is read out from an external storage device storing the compressed image in the range direction, and each line in the azimuth direction is read out from the storage device of the processing device. The rearrangement process is repeated so that the pixels have consecutive addresses.

The above is an overview of the image reproduction algorithm for SAR received image data.
A flowchart comparing the shape of SAR received data is shown in Figures 2A and B. A shows a flow diagram, and B shows a schematic diagram of image processing corresponding to the processing flow.

Conventionally, this series of image processing was performed using a general-purpose computer, but this required FFT and IFFT once in each of the range and azimuth directions for each pixel, and also required a huge amount of rearrangement processing called corner turning processing. For this reason, SAR imaging used to take a very long time. By the way, it takes 20 to 30 hours on a general-purpose large-scale computer to image 100 km square of SAR data received from a certain satellite. This time can be shortened by using a very high-speed computer, but since the amount of processing is incomparably greater than that of conventional image data, the fastest computer possible with current semiconductor technology is required. Even if it is used, it is thought that the limit is to shorten the time to about one hour.

This is an insurmountable barrier because the processing must be done serially when a general-purpose computer is used to program the processing algorithm. It has been impossible to process the data in real time using conventional methods.

An object of the present invention is to provide a dedicated SAR imaging data processing device that performs SAR imaging processing at high speed in accordance with a SAR processing algorithm.

The present invention allocates corresponding processing devices to each processing step necessary for SAR imaging, operates them in a multi-stage pipeline, and also allows the pipelined units to operate in parallel. For corner turning processing, which was the most time-consuming process, we used a two-dimensional addressing image memory to eliminate the need for program-based rearrangement processing, which enabled high-speed processing of SAR imaging that could not be achieved with conventional technology. A feature of the present invention is that it provides a real-time reproduction device for SAR images that enables.

FIG. 3 shows an example of a concrete configuration of a SAR imaging device corresponding to FIG. 2.

The SAR imaging device is centered around a CPU 500 that manages the entire system, a magnetic tape on which SAR received image data is recorded, a playback device 505 for the magnetic tape, a range direction compression device 510, a two-dimensional image memory 10,
10', an azimuth direction compression device 520, a magnetic tape 505' for outputting SAR image data configured to flow without passing through the CPU, and a main memory 530.
and the FFT device 21 for common data on the data bus 5.
40 to form a configuration.

Next, the operation of each device shown in FIG. 3 will be explained in comparison with FIG. 2.

(1) First, under the control of the CPU 500, the magnetic tape 5 is
05 (here, a magnetic tape is considered as the input storage medium) is operated to reproduce the SAR received image data. In the SAR received image data,
It contains hologram type image data for actual imaging, orbit and attitude data of satellites and aircraft, the former is sent to the range direction compression device, and the latter is sent to the range direction compression device.
It is read into the main memory 530 via the CPU.

(2) The range direction compression device 510 sequentially inputs the L×M data in FIG. 2 as data for imaging. The parallel processing control unit 511 performs control so that when performing parallel operations during range direction compression, the processing is distributed for each processing unit, and the operations of the compression devices arranged in parallel proceed in parallel.

(3) The hologram-type image data distributed by parallel operation control is first converted from integers to a plurality of numbers by the preprocessing device 512. (Figure 2, Block B) (4) Next, for compression in the range direction, the complex hologram type image data is subjected to Fourier transform by the FFT device 513 to obtain frequency domain data. (Block C in Figure 2) (5) On the other hand, the reference function obtained in advance by the CPU is also subjected to Fourier transformation by the FFT device 21 to obtain data in the frequency domain. (Blocks P and Q in FIG. 2) This data is stored in the reference function memory 514.

(6) Both data obtained in steps (4) and (5) are multiplied by a complex multiplier 515.
(Block D in Figure 2) (7) Next, inverse Fourier transform is performed by 516 of the IFFT device to complete compression in the range direction (second
(Figure block E) (8) The compressed image contains surplus data due to convolution integration in the frequency domain, and the post-processing device 517 performs fraction processing to cut it off. The results are stored in a two-dimensional image memory as range direction compressed data.
(Blocks F and G in Figure 2) (9) The parallel processing control 518 collects again the data that has been compressed in the range direction in parallel with respect to the data distributed in (2) above, and processes the data based on the order in which they were reproduced from the magnetic tape. It has a function to align.

(10) The two-dimensional image memory control unit 519 is coupled to the range compression device, the CPU, and the azimuth compression device, and performs multiple access to the two-dimensional image memory, alternating buffer management, and two-dimensional addressing control.

(11) Before performing azimuth compression using the azimuth compression device, it is necessary to perform data transposition called corner turning on the range-compressed image, but this can be done quickly using the two-dimensional addressing function. can.
FIG. 4 explains the contents.

When image data ABCD is stored, it is scanned in the direction from a to d and stored line by line, and then stored from b to c. Two-dimensional address (I, J) for each pixel
If you have , the storage process can be expressed as follows. (Formula (1)) (1, 1) (1, 2)... (1, J (2, 1) (2, 2)... (2, J) 〓 (I, 1) (I, 2) ...(I, J) ...(1) When reading out the next time, the two-dimensional image memory 10 is scanned in the direction a → b, and one line at a time is read out sequentially from d → c. This process is performed in the two-dimensional image memory 10. Expressed using address (I, J), (1, 1) (2, 1)... (I, 1) (1, 2) (2, 2)... (I, 2) 〓 (J, 1 ) (J, 2) ... (I, J) ... (2), and when stored, it was image data abcd, but after reading it, it became image data abcd, and it became two-dimensional data that was easily transposed. (Figure 2 Block I) (12) When one two-dimensional image memory becomes full,
Azimuth compression is performed on the data.

At this time, the range direction compression device can continue range compression of the next data using the alternating buffer function of the two-dimensional image memory.

By the time range compression is completed, the CPU uses the trajectory and attitude data read from the magnetic tape to calculate the Doppler shift coefficients necessary for azimuth compression, and also uses the FFT device 21 to perform Fourier transformation of the reference function. Go,
Before performing azimuth compression, the data shall be stored in the Doppler shift coefficient memory and the reference function memory respectively (blocks S, T, and U in Figure 2). Read sequentially from memory.
When parallel operations are performed during azimuth direction compression, the parallel processing control unit 521 performs control so that the operations of the compression devices distributed in processing units and arranged in parallel proceed in parallel.

(14) The range compressed data distributed by the parallel operation control section is subjected to necessary preprocessing by the preprocessing device 522.

(15) The preprocessed image data is subjected to Fourier transformation by the FFT device 524 for compression in the azimuth direction, and is converted into frequency domain data. (Block J in Figure 2) (16) Next, based on the contents of the Doppler shift coefficient memory 523, the range curvature correction device 525
Correction is made by The content of range car drift correction is mainly interpolation based on the interpolation method, and this is explained in the "Data resampling device".
(Utility model registration application 126737, 1984, Utility model registration 1987-
50756), so the explanation will be omitted.

(17) Since the reference function memory 526 stores coefficients determined in advance, the complex multiplier 527 multiplies the data determined in step (16). (Figure 2 block K,
L) (18) Next, an inverse Fourier transform is performed using the IFFT device 528 to complete the compression in the azimuth direction. (Figure 2 Block M) (19) As with range compression, there is surplus data associated with convolution in the frequency domain in the compressed image, and it is necessary to perform rounding to cut it off and actual data. A post-processing device 529 performs real number conversion and integer conversion processing to create image data. (Second
(20) The parallel processing control unit 521' performs the step (13)
It has a function of re-collecting the azimuth-compressed data in parallel with the data distributed in , arranging it as image data that can be visually understood, and outputting it to an output storage medium (magnetic tape, etc.) 505'.

The above is a description of the configuration and operation of the SAR imaging device shown in FIG. 3, and now the operation as a hardware in the range direction and the azimuth direction will be further explained.

The operation of the hardware according to the present invention will be explained in correspondence with FIG. FIG. 5 is a diagram showing an embodiment of the hardware configuration of the range direction compression device in FIG. 3. In this figure, the pretreatment device 51
Reference numeral 2 denotes a preprocessing device that performs functions such as read control of the magnetic tape device 505 and serial/parallel data conversion, which serve as input data to the data processing device. The line buffer 22-1 is a buffer memory for storing data processed by the preprocessing device, and two sets of line buffers 22-1 are provided to function as alternating buffers. This configuration is based on the line buffer 22-
2, 3, and 4 also have similar functions and configurations. As is well known, the FFT 513 and IFFT 516 are a fast Fourier transform device, and the latter is an inverse transform device. Further, the complex multiplier 515 is a complex multiplier based on the constant table 514. The constant table 514 may be implemented as a RAM or ROM table. Finally, post-processing device 517 functions as an image memory control device. 1
It recognizes the breaks in each line and sequentially writes them into the image memory.

Next, FIG. 6 is a diagram showing an embodiment of the azimuth direction compression device in FIG. 3. This device is SAR
This is hardware that performs vertical restoration processing for all image data of the restoration device. In this figure, the preprocessing device 522 functions as an image memory readout control device that reads out all image data in the vertical direction from the image memory in which all image data is compressed in the horizontal direction and stored. . FFT524, IFFT
528, line buffer 22-5, 22-6, 2
2-7, 22-8, 22-9 in Figure 5
This is hardware that performs the same functions as FFT, IFFT, and line buffers.

The resampling device 525 performs resampling processing based on the weighting constant W, and the complex multiplier 527 performs complex multiplication based on the conversion table 526 into which the output of the FFT device 21 is input via the data bus. The constant table device 523 is realized as a RAM table memory. Post-processing device 529
is a write control device for vertically restored data to a magnetic tape device. The division of each line in the vertical direction is recognized and written into the image memory one by one. FIG. 7 is a detailed model of the FFT device shown in FIGS. 5 and 6. In explaining this figure, although it is common knowledge, we will use the Fourier transform.
Let me briefly explain the concept of FFT. Fourier transform, as is well known, is a method of obtaining a spectrum from a signal waveform. In this transform, especially when a finite number of samples are targeted, including discrete time and frequency samples, this is called discrete Fourier transform. It is called. _{In this case ,} ^the discrete Fourier transform can ^be written as _a ^formula _{: k}・W ^-nk _N. Here, W _N = e ^-j2 〓 ^/N・X _o (n = 0 to N-1) is the sample of the time function obtained by sampling the signal at equal intervals X _k (k = 0 to N-1) indicates N-point samples of the frequency domain function.

As is obvious from the above equation, the Fourier transform,
If the inverse transformation is properly performed on N points, N ²
A calculation algorithm called FFT attempts to reduce the required complex multiplications to N/2log ₂ N complex multiplications. Among these algorithms, a method based on the time-varying method is shown in FIG.

In the FFT operation signal flow diagram shown in Figure 8, there are 8 points.
The FFT signal flow is shown as an example.

In the real-time SAR image reproducing device to which the present invention is applied, this image data is approximately
It is thought to have 14,000 pixels and approximately 13,000 lines in the vertical direction. Therefore, when raised to a power of 2, it becomes 2 ¹⁴ , which is the 8th
The number of stages in the figure is 14. (3 stages in the example in Figure 8) Here, Figure 9 shows the basic arithmetic circuit of FFT.
A butterfly calculation circuit A and its calculation formula B are shown.

As is clear from the signal flow diagram in Figure 8,
FFT can be realized by repeating this butterfly operation. FIG. 7 is a diagram modeling the FFT according to the present invention. FFT1 to FFT14 correspond to this butterfly calculation circuit. Theoretically, it is said that approximately N/2 butterfly arithmetic units are required for each stage of FFT. Considering this using the SAR image processing system mentioned above, 2 ¹⁴ /2≒8000 units would be required. However, this would require an enormous amount of hardware, so the FFT hardware of the present invention is as shown in Figure 7.
Each stage has one butterfly arithmetic unit, and the butterfly arithmetic unit equivalent to 8000 units can be performed using the serial arithmetic system.

Therefore, alternating buffers are provided before and after each butterfly computing unit. FIG. 10 shows an example of the hardware configuration of the butterfly computing unit provided in each stage.
In the figure, the sequencer 25 is the address generator 2.
4, the operand is discharged from the alternating buffer 23-4 and supplied to the butterfly arithmetic unit 26.

At the same time, the coefficients specific to the FFT calculation are read from the coefficient storage memory 27, thereby performing the desired butterfly calculation, and transmitting the results to the alternating buffer 23-5 for the next stage.
This means that it will be stored in . This calculation is repeated for each alternate buffer with a capacity of 16K points. In addition, line buffer memories 15-1 to 15
~9 is also the same as replacement batshua 16K x 43 x 2 = 128KB
Two sets of capacity will be provided.

Further, IFFT can be realized by the same means as FFT, so a description thereof will be omitted.

Although the present invention is a SAR imaging device, it is fundamentally the same as digital reproduction of hologram-type images as well as SAR, and can be applied to things other than SAR.

In addition, the range direction compression device and the azimuth direction compression device are based on FFT and IFFT, and the only difference is in the part where the resampling device and the complex multiplier are connected, so they are ORed and the azimuth direction compression device is redundant. It is also possible to use one compression device in both the range direction and the azimuth direction. In this case, the throughput is reduced to about half, but the amount of hardware is reduced, making it possible to create an economical system.

According to the present invention, imaging of SAR received data, which conventionally took more than an hour per scene even using an ultra-high-speed computer, and several tens of hours using a normal large-scale computer, can be done in about 10 minutes or less. This makes it possible to perform real-time imaging using an all-digital system, which is currently considered extremely difficult.

In the field of remote sensing, it is expected that SAR will become the mainstream sensor in the future, replacing various current sensors, and the SAR imaging device according to the present invention will become a very important system.

[Brief explanation of drawings]

Figure 1 is an overall configuration diagram of SAR image processing.
It shows the positioning of SAR, ground photography range, receiving stations, etc. FIG. 2 is a flowchart representing the SAR imaging processing algorithm, and also describes the processing procedure for received image data. FIG. 3 is a block diagram of a SAR imaging device according to the present invention. FIG. 4 is an explanatory diagram of the operation of the two-dimensional image memory in FIG. 3. FIG. 5 is a block diagram of the range direction compression device in FIG. 3. FIG. 6 is a configuration diagram of the azimuth direction compression device in FIG. 3. FIG. 7 shows the configuration of the FFT device used in each compression device in the range direction and the azimuth direction. FIG. 8 is a signal flow diagram of FFT.
FIG. 9 shows the FFT butterfly calculation circuit and calculation formula. FIG. 10 shows an example of the hardware configuration of the butterfly arithmetic unit in each stage of the butterfly arithmetic circuit shown in FIG. 510... Range direction compression device, 513...
FFT device, 516...IFFT device, 10,10'
... Two-dimensional image memory, 520 ... Azimuth direction compression device, 522 ... Preprocessing means.

Claims (1)

[Scope of Claims] 1. In a synthetic aperture radar image processing system that performs processing for reproducing image data obtained by the synthetic aperture radar as an image, at least range direction compression processing of image data obtained by the synthetic aperture radar is performed as parallel processing. a two-dimensional image memory device that stores output data from the range-direction compression processing device; and an azimuth-direction compression processing device that performs azimuth-direction compression processing of the output data from the two-dimensional image memory device as parallel processing. a compression processing device; a transmission path through which the imaging data and each output data are transmitted; a data bus line through which control signals for controlling each of the devices are transmitted; and a control signal for controlling each of the devices.
and a CPU, the two-dimensional image memory device inputs output data from the range direction compression processing device to at least two two-dimensional image memories and one of the two-dimensional image memories in parallel. , the other above 2
Outputting from the dimensional image memory to the azimuth direction compression processing device, and controlling the scanning direction of the two-dimensional image memory during input and the scanning direction of the two-dimensional image memory during output to be different.
An image processing system for a synthetic aperture radar, comprising: a dimensional image memory control section.