JPS63219082A - parallel image processing processor - Google Patents

Abstract

PURPOSE:To form an architecture suitable to LSI by providing an image data input port, plural shift registers, an arithmetic circuit, etc., and giving a product sum summation memory commonly to plural processor units. CONSTITUTION:When a product sum summation memory 15 is commonly given to processor units PE12#1-#4 and an image f14 is inputted from an image data input port 24 at time 1, images f11-f14 are given through a shift register 11 to respective PEs. A load W11 is read from the memory 15, the product with the input image is obtained and held at a cumulative arithmetic circuit 20. Next, an image f15 is inputted at time 2, images f12-f15 are given to respective PEs, the product with a next load W12 is obtained and the cumulative processing with a previous value is executed at the circuit 20. Thereafter, the same processing is executed in accordance with respective times, and a space product sum arithmetic result is continuously outputted through a partial sum output shift register 21 and a partial sum cumulative arithmetic circuit 14. Thus, the architecture suitable to the LSI can be formed.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a parallel image processing processor that performs local neighborhood image processing such as spatial product-sum operations, and particularly to a parallel image processing processor having an architecture suitable for LSI implementation.

Image processing processors process image data in parallel at high speeds, as developed in the Ministry of International Trade and Industry's large-scale project "Pattern Information Processing System" (a collection of research and development results was published in October 1980). There are many things that we are trying to change.

Since image data has a two-dimensional spread, it is difficult to process all image data in parallel. However, since there are many calculations between neighboring image data, such as spatial product-sum calculations that realize noise removal and contour extraction functions, for example, it is difficult to process local data in m rows by n columns of an image in parallel. many.

This type of locally parallel image processing is described in the above-mentioned literature or Masatsugu Kido, Trends in Image Processing Hardware: Information Processing Computer Vision Study Group Material 8-6 (September 1'980).
Although it has been comprehensively explained in 2011, there is no LSI version except for the CCD analog processing type. There are difficulties in converting a processor with a conventional architecture into an LSI as it is in terms of: 1) the degree of integration, and 2) the number of pins.

An object of the present invention is to provide a parallel image processing processor having an architecture suitable for LSI integration.

A feature of the present invention is that in a parallel image processing processor that takes in image data from an image data source and performs local parallel image data processing, it includes an image data input port and a plurality of parallel image processing processors that sequentially take in image data from the image data input ports. a first shift register, a plurality of processor elements that input the contents of each of the first shift registers and perform image processing operations, and cumulatively add the operation results in each of the processor elements for each processor element. a plurality of first arithmetic circuits, a second shift register that receives the arithmetic results of the plurality of first arithmetic circuits, and an arithmetic result data input port that inputs the arithmetic result data of the preceding basic module; a second arithmetic circuit that adds the arithmetic result data and the arithmetic result of the first arithmetic circuit set in the second shift register; and an arithmetic operation that outputs the arithmetic result data of the second arithmetic circuit. The parallel image processing processor includes a plurality of sets of image processing processor basic modules each consisting of a result data output port and a result data output port arranged in parallel.

Hereinafter, the present invention will be explained using illustrative embodiments. still,
1 to 10 are explanatory diagrams of recently considered parallel image processing techniques, and FIGS. 11 and 12 are one embodiment of the present invention.

FIG. 1 shows the configuration of a typical image processing system, which includes an industrial television camera 5 as an image input device, an image memory 3 as an image storage device, and a CRT monitor 4 for displaying the contents thereof. The image information in the image memory 3 is processed by the image processor 2, and the results are also stored in the image memory 3 or provided to the management processor 1 which controls the entire system.

A typical image processing function is spatial product-sum operation. As shown in FIG. 2, this is, for example, 4×4 pixel local image data f! 1 to f44, the determined loads Wll to W
44 and calculates the sum.

This results in noise removal Contour enhancement Image processing such as

As an image processing processor that processes local image data of 4×4 pixels, for example, a 4×4 image processor as shown in FIG.
A parallel image processing processor (referred to as type I) that combines four image processing processor basic modules 10 each having 12 processor elements (PE#1 to #4).
2-I. One column of local image data (f14 to f44 in FIG. 3) is given in parallel from the image memory 3, and the calculation result (g in FIG. 3) is stored in the image memory 3.

The basic module 10 has an image data input port 24 that takes in image data of a row to be processed, and a calculation result data output port 35 that outputs internal processing results. image data f
When 14 is input, 1 is input through shift register 11.
Pixel by pixel adjacent pixel f 11 f sx, f 11
is also input to the corresponding PE#4-1. Pixel fzt is output from the image data output port 25 in case the size of the spatial product-sum operation is expanded to 4×4 or more. P
The image data f to be processed from the shift register 11 and the load data W from the load storage memory 15 are given to E12, and multiplication by 1 is executed. This result is 4 PE1
A partial sum is calculated by the arithmetic circuit 13 which adds the results of the two results. Partial sums inputted from the calculation result input port 30 are accumulated one after another by the calculation circuit 14, and outputted from the calculation result output port 35 to the basic module 10 at the next stage.

In this way, by stacking the basic modules 10 in four stages, the final basic module IOD outputs.

This time chart is shown in FIG. The above-mentioned operation is executed within the basic clock time Δt1 and the result g is output,
At the next Δt1, the result g for the input image of 4×4 picture elements shifted by one pixel is output. Therefore, they are input one after another.

The spatial product-sum calculation results of all 4×4 picture elements for the image data are output one after another.

The embodiment shown in FIG. 5 shows a configuration in which the basic clock time Δt1 of the type 1 image processing processor 2-I of the previous embodiment is shortened by pipeline processing. This is called a pipeline version parallel image processing processor 2-IP of type (2). That is, in Type I, the basic clock time Δt1 is: ■ Input processing of image data f+, j to the shift register 11 ■ Product-sum load Wl, J by the processor element 12
Multiplication processing of images fl and J by image fl, J; Partial sum processing by arithmetic circuit 13; Partial sum accumulation processing by arithmetic circuit 14.

On the other hand, for example, as in the example in Figure 5, ■, ■, ■
By interposing the pipeline register 16 between and ■, and between ■ and ■, the basic clock time Δt2 can be changed to ■
The maximum processing time of ~■ (not the sum of all)
It is possible to make it as small as possible. This time chart is shown in FIG.

Processing ■ is executed at time 1, ■ at time 2, ■ at time 3, and ■ at time 4. At time 2, the next input image is processed ■.

3, 4, and 5 are executed, and the processing speed can be improved by operating each component one after another in a pipeline manner.

The embodiment of FIG. 7 is based on the parallel image processing processor 2-
This shows a configuration in which the IP basic clock Δt2 can be further shortened, and is called a type (2) pipeline-skew version parallel image processing processor 2-IPS. Fifth
The basic clock time Δt2 in the IP type shown in the figure is the processing ■
There is a strong possibility that it is constrained by the partial sum accumulation time of . This is because when the basic module 10 has n stages, Δt2 requires n times the sum of the processing time in the arithmetic circuit 14 and the input/output time of the arithmetic result 30.35. In particular, when the basic module 10 is implemented as an LSI, the input/output delay time cannot be ignored. For this reason, a pipeline register 16 is further added to the type tp shown in FIG. The time regulation is set to 1/n. In the IPS type shown in FIG. 7, as shown in the time chart of FIG. 8, the partial sums of the basic modules IOA to D are calculated at the same time 3, and the timing of the cumulative part does not match. In the IPS shown in FIG. 7, a variable stage skew correction shift register 17 for timing adjustment is installed immediately after the image data input port 24. Since the number of pipeline stages in the cumulative path of each basic module l0A-D is one stage, the number of stages of the variable stage number skew correction shift register 17 is as follows: basic module IOA...O stages B. ...
...1st step C...2nd step D...3rd step. In this way, the mismatch (... part) in the time chart of FIG. 8 is corrected, and the continuous Δt
Pipeline operation can be completed in 3 hours.

As can be easily seen, the timing mismatch is similarly resolved even if the skew register 17 is installed immediately after the arithmetic circuit 13 that calculates the partial sum, or even if it is installed immediately before or after each PE 12. .

FIG. 9 shows another embodiment with a different processing form.

In the configuration of type (2) described above, image data input is distributed to adjacent picture elements to each PE 12 #1 to -4 via the shift 1 register 11. In contrast, in this embodiment, the input image data is commonly given to each PE 12 #1 to #4, and the multiplication results are accumulated via the arithmetic circuit 18° register 19 to output the partial sum Σ1. . This operation will be explained with reference to the time chart of FIG.

At time 1, the image flt is input from the image data input port 20, and the product f11-Wll with the load Wll read out from the load storage memory 15 at PE12#1 is stored in register 1.
9 #2 is set.

Image data fxz is input at time 2, and PE 12#2 takes the product f1z*wzz with load wt2, and calculates the sum f11* of this and the value fl1mW11 of register 19#2.
w1t+ft2*W12 is taken by the arithmetic circuit 18 and set in the register 19#3.

Image data fza is input at time 3, and the product f1s-A13 with the load W13 is taken at PE12#3, and this and the value of register 19#3 f11 * wtt + f12
Japanese full umbrella with mock W12 Wll + ftz umbrella W12 + f
The ta umbrella W13 is taken by the arithmetic circuit 18, and the register 19
Set to #4.

Image data fx4 is input at time 4, and PE12#4 takes the product ft4 umbrella W14 with load W14, and combines this with the value f1 of register 19#4 to obtain w1t+fx2newxz+
Sum of ft3*wta Σ11=fli book Wll 10~+f
x4*wta is taken by the arithmetic circuit 18. This partial sum Σ
1 is accumulated in the arithmetic circuit 14 of each basic module l0A-D, and is output from the final stage.

Thereafter, the spatial product-sum calculation result g is output at intervals of each basic clock time Δt4.

This type H parallel image processing processor 2-n also has type! Similarly, types ■P and ■PS can be considered, and it is possible to reduce the basic clock time Δt4.

Since these and others can be easily inferred, they will be omitted here.

FIG. 11 shows an embodiment of a parallel image processing processor according to the present invention. In contrast to the method described above in which a sum-of-products load (memory) 15 was given to each PE 12 independently, the configuration shown in Fig. 11 is a method of giving a sum-of-products load (memory 15) to all PEs 12 in common, and is a parallel type of type ■. Image processing processor 2-m
It is called. This operation will be explained with reference to the time chart of FIG.

First, at time 1, the image f has already been input from the image data input port 20.
Assume that 14 is input. At this time, fttt is sent to PE12 #1 to #4 through the shift register 11.
ftz, f□3. The f1 number is given. Then, the load Wit is read out from the load storage memory 15 and multiplied by each input image. In the arithmetic circuit 20,
The value held at the beginning of time 1 is cleared to 1'0'', and the products of the aforementioned f11 to f14 and Wll are held respectively.

At time 2, image f15 is input, and PE12 #1 to #4
are respectively given f12 to f15, and the next load W12
Which product is taken? Thereafter, the arithmetic circuit 20 performs an accumulation process with the previous value. For example, λ is A1 and 2 is f 111 W
1! + f 121 A12, f 121 w in A2
1t+f 1s's A12 is retained as the result.

The same process is executed at times 3 and 4, and the arithmetic circuit 2o#
A1 :Σjl:: f tt * wtt + f 1 for 1 to #4
2F W12+ f 13” w1a+ f 14” W
14#2:Σi2:=4 tz* A11+ f 13
1 W12 + f14 monkey w1a + f xe, * W14
#3: Σlδ= f la * wtt + f 146
A12+ f 15* wta+ f 1B” W14
#4: Σea=ft4 monkey W11+ f ts$ W12
10 ft 8 wia + f 1? -WL4 and the respective first partial sums are obtained, which are set in the shift register 21 at the end of time Δ.

At times 5 to 8, Σif to Σ11. Σ)2 to Σ12.
Σ)3 to Σ13. Σi4 to Σ141 are sequentially accumulated by the arithmetic circuit 14, and the results gll to g14 are output.

At the same time, in PEA1, image data fxδ~f16°P
fza~fto in EA2, f17~fz in PEA3
o, in PEA4, time 1 to f ha to f zl
4 is executed, and the partial sum Σ15. Σ1G.

Σi7. Σ18 is calculated and these are accumulated at times 9 to 12 to obtain results g16 to g1δ. In this way, spatial product-sum calculation results are continuously output.

Similar to the type (2), types (2) P and (2) PS can be considered for the parallel image processing processor 2-III of the type (2), and it is possible to reduce the basic clock time Δt5.

Now, in the embodiments of types (1) to (4) described above, calculations between the basic modules 10 are performed by connecting partial sum calculation circuits 14 in series, and this circuit 14 is also included in the basic module. However, if the number of pins becomes an issue for LSI implementation, for example, only the dotted line in Fig. 3 is set as the basic child Joule.
Inter-module operations can also be performed externally in parallel.

According to the present invention, one locally parallel image processor can be divided into modules with a small number of input/output ports and a regular arrangement, so that an architecture suitable for LSI implementation can be achieved. In particular, since the product-sum load is commonly applied to each processor element, one RAM for storing the load coefficients can be used in common, only one port is required, and it is easy to manufacture as an LSI.

[Brief explanation of the drawing]

Figures 1 to 10 are explanatory diagrams of recently considered parallel image processing techniques, with Figure 1 showing the configuration of an image processing system, and Figure 2 showing an example of local parallel processing. , FIG. 3, FIG. 5, FIG. 7, and FIG. 9 are block diagrams showing configuration examples of parallel image processing processors, and FIG. 4, FIG. 6, FIG. 8, and FIG. 10 are block diagrams showing configuration examples of parallel image processing processors. FIG. 11 is a diagram of an embodiment of a parallel image processing processor according to the present invention, and FIG. 12 is a time chart of FIG. 11. 2... Parallel image processing processor, 3... Image memory. 10... Image processing processor basic module, 11.
... Input image shift register, 12... Processor element, 13... Partial sum calculation circuit, 14... Partial sum accumulation calculation circuit, 15... Load storage memory, 16...
- Pipeline register, 17... (variable number of stages) skew correction shift register, 18... Propagation/accumulation calculation circuit, 19... Propagation register, 20... Accumulation calculation circuit, 21... Partial sum output Shift register, 24... image data input port, 25... image data output port. 30...Arithmetic result data input port, 35...Arithmetic result No. 1 Fig. Z Fig. 3, f54c 2u > tz ... f+ 6 Fig.) II Jrz No. grb Fig. 9 Fig. 1 O'in 2rz Figure l1 Figure 1Z

Claims (1)

[Claims] 1. In a parallel image processing processor that takes in image data from an image data supply source and performs locally parallel image data processing, an image data input port and a plurality of first processors that sequentially take in image data from the image data input port are provided. a shift register, a plurality of processor elements that input the contents of each of the first shift registers and perform image processing operations, and a plurality of processor elements that cumulatively add the operation results in each of the processor elements for each processor element. 1
an arithmetic circuit; a second shift register that receives the arithmetic results of the plurality of first arithmetic circuits; an arithmetic result data input port that inputs the arithmetic result data of the preceding basic module; a second arithmetic circuit that adds arithmetic results of the first arithmetic circuit set in the second shift register; and an arithmetic result data output port that outputs arithmetic result data of the second arithmetic circuit. A parallel image processing processor characterized in that a plurality of image processing processor basic modules consisting of the following are installed in parallel.