JPH09120443A - Image processing method and apparatus - Google Patents

Abstract

(57) Abstract: An image processing method and apparatus capable of separating attributes of an input image and generating statistical information thereof to facilitate subsequent image retrieval. SOLUTION: An image is input (1, 2), the input image is separated into partial images having a predetermined attribute (4), statistics of the generated partial images having the same attribute are taken (5), and the partial images are generated. An image of the statistical result of the partial image thus formed is formed (6).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and an apparatus therefor, and more particularly to an image processing method and an apparatus for separating an input image from attributes and generating statistical information of the input image to form an image.

[0002]

2. Description of the Related Art In a conventional copying machine, an image is input by a scanner, and various image processing such as noise removal and quantization is performed on the input image to form an image.

[0003]

However, in recent years,
In the copying machine, it is necessary to extract not only the copy function but also the attribute information of the input image and use it as a search key to search for the corresponding image. The present invention has been made in view of the above-mentioned conventional example, and provides an image processing method and an apparatus thereof for separating attributes of an input image, generating statistical information thereof, and facilitating subsequent image retrieval. To aim.

[0004]

In order to achieve the above object, an image processing method and apparatus according to the present invention have the following arrangement. That is, an image input step of inputting an image, an attribute separation step of separating the image input in the image input step into partial images of predetermined attributes, and a partial image of the same attribute generated in the attribute separation step. A statistical processing step of collecting statistics and an image forming step of forming an image of a statistical result of the partial image generated in the statistical processing step are provided.

Another aspect of the invention is image input means for inputting an image, attribute separation means for separating the image input by the image input means into partial images having a predetermined attribute, and generation by the attribute separation means. The statistical processing means for obtaining the statistics of the partial images having the same attribute, and the image forming means for forming the image of the statistical result of the partial images generated by the statistical processing means.

[0006]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail. FIG. 1 is a block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention. In FIG. 1, 1 is a scanner unit that captures an image, 2 is a memory control unit, 3 is a memory unit, 4 is an attribute separation unit that performs attribute separation according to the characteristics of the read image information, and 5 is
A creation unit 6a that creates a list of classification information that summarizes information such as the classification content and quantity of each image element based on the attribute separation result.
Is an output unit for outputting the list created by the creating unit 5, 6
Reference numeral b is a display unit (control panel), and reference numeral 7 is a system control unit that controls the system of the image processing apparatus.

Next, the overall operation principle will be described with reference to FIG. The scanner unit 1 scans a document to read a document image, and the document image is stored in the memory unit 3 via the memory control unit 2 controlled by the system control unit 7. The image information read by the memory control unit 2 is supplied to the attribute separation unit 4 and is separated into image elements (character portion, title portion, photograph portion, etc.) according to the characteristics of the read image information. After that, the attribute separation result is supplied to the creation unit 5. At this time, at the same time, a list creation control signal is supplied from the system control unit 7 to the creation unit 5, and according to the control signal of the system control unit 7, the classification name of each image element is obtained from the attribute separation result sent from the attribute separation unit 4. Then, a classification information list is created as classification information in which information such as quantity (for example, title part = present / absent) is collected.

FIG. 2 shows a processing example of the above processing. 20
3 is a screen showing an example of the result of separating the input image into image elements by the attribute separating unit 4. That is, an example in which the title part and the front part surrounded by each square are separated is shown. Then, this separation information is input to the creation unit 5, and a classification information list of classification names and the number thereof as shown in 21 is generated. Images corresponding to this classification information list are stored in the built-in memory 8.

After that, it becomes easy to search for a corresponding image by using the attribute name of this classification information list. Then, the list of the classification information is output to the output unit 6. When the list output control signal is supplied from the system control unit 7 to the output unit 6, the printer output unit 6a prints out the input classification information.

Next, FIG. 3 shows an example in which classification information is generated for a large amount of image information 31 and is printed out. In this example, the attribute separation unit 4 extracts the separation information from the large amount of image information 31 collectively, and the creation unit 5 generates the classification information list 31 including the classification name and the number thereof. In this classification information list 31, each classification information (classification name and its number) corresponding to the reference number (or page number) of each image information is arranged in order.

The display section 6b has a built-in display,
The result of attribute separation of image information is displayed. This display portion 6b
May be integrated with the control panel of the image processing apparatus of the present invention. Next, the attribute separating unit 4 will be described in detail. The attribute separation unit 4 is a processing unit that performs attribute separation of image data to create area information.
The operation principle of is described below with reference to FIGS. 4 to 27.

FIG. 4 is a flowchart showing a rough process in the attribute separating unit 4.

In order to increase the attribute separation processing speed in the attribute separation unit 4, the image data is thinned out in step S1. When the thinning-out of the image data is performed, the processing of the attribute separating unit 4 is performed on the thinned-out image. The thinning out of this image data is mx
This is done by examining the connectivity of black pixels in the m pixel block. For example, in a 3 × 3 pixel block, when there are two connected black pixels, the 3 × 3 pixel block is thinned out to one black pixel. On the contrary, when there are two connected white pixels in the pixel block, the pixel block is thinned out to one white pixel.

Next, in step S2, the image of the pixels is analyzed, the connectivity between the pixels is searched, and the pixels are classified according to their size and relative position to other connected components. A connected component is a set of black pixels completely surrounded by white pixels. Therefore,
One black pixel connected component is completely separated from other black pixel connected components by at least one white pixel.

The process of step S2 will be described later with reference to the flow charts of FIGS. 5 to 7, but the outline thereof will be described. Search for a connected component and information obtained from its size and some connected components are obtained. The connected components are classified based on the statistical information of. In this classification, each connected component is first classified into a text unit or a non-text unit. The non-text unit is then analyzed in more detail to determine if it is data with a frame structure, halftone image, line drawing, table, or other tabular text data. If the data structure is unknown, it is not classified as unknown. A hierarchical tree structure is then created for each connected component to provide the constituent data for the connected component and facilitate reconstruction of that data.

Next, in step S3, adjacent connected components are grouped unless a gap line is sandwiched therebetween. The grouping here may be performed in the vertical direction or in the horizontal direction. This corresponds to whether the text units to be grouped are vertical writing or horizontal writing.The distance between connected components that are close to each other in both directions is checked beforehand in each of the horizontal and vertical directions, and the horizontal If the distance in the direction is small, move horizontally,
If the distance in the vertical direction is small, grouping is performed in the vertical direction.

The tree structure generated in step S2 is used to prevent improper mixing of text and non-text. Further, in step S3, it is determined whether the text units are grouped into lines by detecting vertically or horizontally extending gaps between the lines and vertically extending boundaries of non-text units. This row structure is retained in the tree structure by updating the hierarchical tree structure appropriately.

Next, in step S4, step S
If the rows grouped in 3 have a small space in the direction opposite to the direction in which they were previously grouped, they become blocks regrouped in that direction. On the other hand, non-text units are used as boundaries for the image page. Text units between two non-text units are processed separately from other text line units.

Further, in step S4, the non-text units that could not be classified in step S2 are analyzed to determine if they are large font size titles. If they were determined to be titles, their units would have the appropriate attributes attached,
Then, the tree structure is updated. The title included in the image data helps to reconstruct the page.

FIGS. 5 to 7 are process flowcharts showing how to detect connected pixels of pixel image data and classify the connected pixels in step S2 of FIG.

First, in step S11, pixel data included in the image data is searched for by contour line tracking. This contour line tracking is performed by scanning an image as shown in FIG. The scanning of this image starts from the lower right portion of the image indicated by the arrow A and proceeds upward until the right edge of the graphic is encountered. This scan may be sequentially scanned in another direction, for example, from upper left to lower right. When a black pixel is hit, whether or not the adjacent pixel is a black pixel is examined in the order of the arrow direction indicated by 31 in FIG. This black pixel search is called an eight-direction search because it is represented by a vector in eight directions when viewed from the center. Then, if there are adjacent black pixels, this process provides the outer contour of the graphic. Thus, as shown in FIG. 9, the scan in the direction of arrow A hits the point corresponding to the end of the letter "Q" 32. An examination of adjacent pixels is performed in the order indicated by 31 to track the outer contour of the letter "Q" 32. It should be noted that the inner part of the closed contour (the inner part of the letter Q) is not traced here.

In this way, when the contour line obtained by the eight-direction search, that is, one connected component is extracted, the scan proceeds until the next black pixel is encountered. Thus, for example, the object 34 that seems to represent a complete black area is searched in eight directions. Similarly, the handwritten character "non"
The non-text object 35, which is "text", is tracked, and the individual character set objects 36a-36d that form the word "text" are tracked. The scan shown in this Fig. 8 is for all connected components. Continues until is detected.

Next, in step S12, all connected components are cut out into rectangles. In this case, the smallest possible rectangle covering the individual connected components will be drawn. Thus, the rectangle 37 around the object 32 in FIG.
, A rectangle 39 is drawn around the object 34, and a rectangle 40 is drawn around the object 35. Also,
Text objects 36a, 36b, 36c, 36d
The same applies to rectangles 41a to 41d for.

In step S13, all the rectangles obtained in step S12 are positioned in the tree structure. In most cases, the tree structure obtained in step S13 results directly from the root for each object. This is because only the outer contour of the connected component is tracked, not the interior of the closed region. Thus, FIG.
As shown in, the rectangle 37 corresponding to the connected component 32 arises directly from the root of the page. However, the rectangle 40 surrounding the non-text object 35 and the text object 3
When the rectangle is completely included in another rectangle (here, the rectangle 39) like the rectangles 41a and 41b surrounding 6a and 36b, these connected components are included connected components (in this case, the components). 34) becomes a child. In addition, ingredient 34
, Each connected component that has at least one child makes the component itself a “primary child”. In the example of FIG. 9, the component 34 includes its own rectangle 39 as a main child together with the other child components 40, 41a, 41b (see FIG. 10).

Next, proceeding to step S14, each first level connected component in the tree is classified as a text unit or a non-text unit. This classification process is 2
Including one step. In the first step, the rectangle surrounding the connected component is compared to a predetermined size. If the height of the rectangle enclosing this connected component exceeds a predetermined value corresponding to the maximum font size, or if the width of the rectangle is the empirically determined constant value (“5”)
, The connected component is classified as a non-text unit and that unit is given the attribute of "non-text".

Then, in the second step, all remaining units for which no attributes have been given, ie, units not classified as non-text, are based on the statistical size obtained from all remaining connected components. It is compared with the determined value. In particular, the average height of all rectangles that were not considered non-text is calculated. An adaptive threshold is obtained by multiplying this average height by a certain value (generally "2"). All units above this threshold are classified as non-text. On the other hand, units smaller than the threshold value are regarded as text. Each unit is then classified and given the appropriate attributes. The classification by the above two steps undergoes some further processing shown in FIGS. This is described in more detail below.

After all the first level units of the tree structure have been classified as text or non-text, the text unit's children are classified as text, including the major children (ie, themselves). On the other hand, the main non-text children are left as non-text, while the other children are classified as text.

Next, in step S15, the first unit is selected. Next, in step S16, if the unit is text, the process advances to step S17 to select the next unit. Then, steps S16 to S17 are performed until the non-text unit is selected in step S16, and when the non-text unit is selected in step S16, the process proceeds to step S18.

In step S18, it is checked whether the non-text unit has children. For example, in the example of FIG. 10, the non-text unit (rectangle) 34 is a non-text main child (rectangle) 39, and the text rectangle 4.
It has children 0, 41a, 41b.

In step S18, if the unit has a child, the process proceeds to step S19, in which each unit is subjected to filtering for halftone (or gray scale). In halftone filtering, the children are examined and the number below the "noise" size is counted. This "noise size" unit is one whose height is smaller than the minimum font size in the input image data. Then, in step S20, when the number of units smaller than the noise size is larger than half of the total number, it is determined that the unit is a halftone image. As a result, the process proceeds from step S20 to step S24, and the attribute of "halftone (HALFTONE)" is given to the unit. Then, in step S25, the text in the halftone image is examined. That is, the tree structure is modified so that the tree structure is at the same level as the halftone image, not in the halftone image child. Here, if the child's treatment is appropriate, character recognition in the halftone image is also possible. Then, returning to step S17, the next unit is selected, and the same processing as described above is executed.

On the other hand, if the unit is found not to be halftone as a result of the halftone filtering in step S19, then the process proceeds from step S20 to step S21, where the major children of that unit are selected for later processing. Then, the process proceeds to step S22.

If it is determined in step S18 that the unit is a non-text unit and has no child, or if the main child is selected for subsequent processing in step S21, the process proceeds to step S22. The unit is a frame
Get filtered. This frame filtering is to determine whether or not the unit is a frame. The term "is a frame" means that there are a plurality of parallel straight lines having almost the same width and height that form a rectangle surrounding the unit. Here, in particular, the line width of the connected component in each row when viewed in pixel units in the target unit is examined.

In FIG. 11, the non-text unit 42
Includes a connected component 43 having a contour component as indicated by 44. In this example, the line width of the child connected component in the row i is Xi, that is, the distance from the left end 45a to the right end 45b of the contour line 44. On the other hand, in line j, two line widths exist inside the connected component 43. That is, 46a-4
6b and between 47a and 47b. The distance between 46a and 46b, which is the longest line width, is defined as the distance Xj.

In this way, the non-text unit 42
The distance X is calculated for every row n in, and whether the non-text unit is a frame is examined by the following inequality. Here, as described above, Xk represents the longest line width in the k-th row in the connected component, W is the horizontal width of the rectangle 42, N
Is the number of rows, and the threshold value is a value calculated in advance so that a frame can be detected even if the frame is slightly inclined. 1 ° here
In order to allow the inclination of, a threshold value of {sin (1 °) × L + (constant value)} may be used. This constant value is the average height of the characters calculated in step S14. If the above inequality is not satisfied, the unit is determined to be frame data, the process proceeds from step S23 to step S26, and the attribute "frame (FRAME)" is added.
Thus, for example, a judgment such as "frame and table" or "frame and halftone" can be performed on the frame.

From step S26 to step S42, the possibility that the frame data includes a table or a table format is checked, and an inspection for obtaining a white contour in the connected component is performed. This white outline is the step S
It is basically the same as the (black) contour obtained in 11 except that it is obtained by examining white pixels instead of black pixels.

As shown in FIG. 14, the inside of the non-text unit is searched in the direction of arrow B from the lower right to the upper left. When a white pixel is first encountered, a search is performed in the outward direction as indicated by 51 for a white pixel in the vicinity from that point. It should be noted that only the directions indicated by 1 to 4 are required here. As a result, the white contour tracking in the processing here is a search in four directions. This process continues until all white contours have been retrieved. For example, the white contour tracing is to extract the contour portion surrounded by the black line segments 52, 53, 54, 55, and the similar processing is performed by 56.
It is also performed for the inside of the black pixel as shown by. In this way, the above-described scanning in the direction of arrow B is continued until all closed white contours in the non-text object are tracked.

Then, in step S43, the density of non-text units is calculated. This density is obtained by counting the number of black pixels in the connected component and dividing the number of black pixels in the rectangle by the total number of pixels surrounded by the rectangle.

Next, in step S44, the number of white contours in the found non-text unit is calculated. If the number is "4" or more, the non-text image may be a table or text blocks arranged in a table, and therefore the process proceeds to step S47, and the white contour filling rate is calculated. To be done. The filling rate of the white outline indicates the ratio of the area surrounded by the white outline in the non-text image.

That is, in the example of FIG. 14, some white contours are formed entirely of white pixels, as shown by 57 and 59, and some white contours, such as 60 and 61, include black pixel areas inside. Exists. If the white contours are highly filled, then the non-text image is probably a table or a block of text blocks arranged in a table. In order to make this estimation more reliable, it is examined whether or not it has a grid-like internal structure in the horizontal and vertical directions with respect to the white contour of interest. Particularly, in step S49, a white contour having a boundary line that does not cross at least two contour lines in the horizontal or vertical direction is regarded as not on the grid and is recombined. For example, in the example of FIG. 14, the left boundary 6 of the white outline 59 is
2 and the right boundary 63 extend in the vertical direction so as to coincide with the left boundary 64 and the right boundary 65 of another white pixel 60. Therefore, it is judged that these white outlines are arranged in a grid,
It will not be recombined. Similarly, the upper boundary 66 and the lower boundary 67 of the white contour 59 extend in the horizontal direction so as to coincide with the upper boundary 68 and the lower boundary 69 of another black pixel 70. As a result, it is determined that these white contours are also arranged in a lattice, and are not recombined.

FIGS. 15 to 17 are diagrams for explaining the case where white contours are combined.

In FIG. 15, for example, the non-text unit 71 represents an example of a non-text unit including a unit from a halftone image to a binary image. The non-text image 71 includes black pixel areas 72 and white pixel areas 74, 75, 76, 77, 78, 79. Probably, the filling rate of this white pixel area is sufficiently high, so that the process proceeds from step S48 to step S49, and is recombined. First, as shown in FIGS. 15 and 16, first, the upper and lower ends of the white contour 75 are compared with the upper and lower ends of the white contour 77. Since their top and bottom edges do not coincide, white contours 75 and 76 are combined to create a new white contour 76 '. When the filling rate of the white pixels is low, the process proceeds from step S48 to step S55.

16 and 17, the white contour 77
The left and right boundaries of the white contour 78 are compared with the left and right boundaries of the white contour 78.
Since these boundaries do not match, the white contours 77 and 79 are recombined into a new white contour 77 '. This process is repeated horizontally and vertically until no recombination occurs.

As described above, the white contours of the table are difficult to combine, and other than the table, such as a table such as a halftone image or a line figure, is easily combined. Then, in step S50, the recombination rate is calculated. If the recombination rate is high or the number of white contours remaining after the recombination process is less than "4", the process proceeds to step S55, and the non-text unit is a halftone image, as described in detail below. , Is determined to be a line figure.

In step S50, if the recombination rate is not high, or if at least four or more white contours remain, the process proceeds to step S51 and it is determined that the table is the attribute (TAB
LE) is added. Next, in step S52, the inside of the figure newly determined to be a table is examined, and the connected components included therein are searched and classified. The tree structure is updated in step S53 according to the new internally connected component. Next, in step S54, the internally connected component is classified into text or non-text again, and an appropriate attribute is added according to the classification result. This process is the same as the process of steps S12 to S14 already described. Then, the process returns from step S54 to step S17, and the next text unit is selected.

If the white contour filling rate is not high in step S48 or the recombination rate is not high in step S50,
The non-text frame graphic is likely to be a halftone image or a line graphic. Then, whether the unit is a halftone image or a line figure is determined by the average of the horizontal run lengths of the black pixels in the unit, the ratio of white pixels to black pixels, and the density. Generally, a very dark image is determined to be a halftone image, and a white and bright image is determined to be a line figure.

Particularly, when the average run length of white pixels is almost "0" (almost black or mottled image), and the density calculated in step S43 is larger in black than in white (that is, If the density is above a threshold of about 0.5 (this is the first threshold), then the frame unit is determined to be halftone. If the density is not greater than the first threshold, the unit is determined to be a line graphic. Also, if the average run length of the white pixels can hardly be said to be “0” and the average run length of the white pixels is larger than the average run length of the black pixels, the frame unit is determined to be a line figure. However, if the average run length of white pixels is not greater than the average run length of black pixels (ie, black is also the dominant image), then more detailed testing is needed. In particular, when the number of black pixels is much smaller than the number of white pixels (that is, when the number of black pixels is smaller than twice the number of white pixels (this is the second threshold)), this frame unit is halftone. Is determined. On the other hand, if the value obtained by dividing the number of black pixels by the number of white pixels is not greater than the second threshold value, but the density calculated in step S43 is greater than the first threshold value, the frame unit is a halftone image. To determine. Otherwise, it is determined to be a line figure.

When it is determined in step S55 that the frame unit is a line figure, the process proceeds to step S58, where an attribute (LINE-DRAWING) called "line figure" is added, and in step S59 all children are removed. To be done.
In particular, once a unit is determined to be a line figure, character recognition processing is not performed for that unit.
Then, the process returns to step S17, and the next text unit is selected.

On the other hand, if it is determined in step S55 that the frame unit is not a line figure, the process proceeds to step S56, an attribute (HALFTONE) of "halftone" is added, and then in step S57, a child of the unit is added. Among them, the text size child is removed. And all children larger than the text size are allowed to remain as children of the frame halftone image. And step S1
Returning to 7, the next text unit is selected.

Next, returning to step S44 of FIG. 6 again, if the number of white contours is not larger than "4", it is determined that the frame unit is not a table and step S45 is performed.
And the density calculated in step S43 is compared with a certain threshold value (about 0.5). This threshold is chosen because the text units and line graphics in the frame should be less than half of all pixels. If this density is less than this threshold, the process proceeds to step S46, and the internal structure of the frame unit is examined. This process is performed by the above-described step S1 for the internal structure of the frame unit.
This is the same as the processing of 1.

On the other hand, in step S45, if the density is larger than the predetermined threshold value, the process proceeds to step S35, and the frame unit is classified as either a line figure or a halftone image, or the frame is It is determined whether it cannot be classified (that is, the frame is “unknown”).

Returning to step S23 in FIG. 5, step S
Step 22 if no frame is detected in the non-text unit by frame filtering 22.
Proceed to 27 to determine if the non-text unit contains a "line". A "line" is a non-text unit useful for delineating text boundaries. But,
The text delimited (enclosed) by such a line is often very close to that line and may be touching. As a result, line searching needs to consider both when the text touches the line and when it does not.

For line searching when no contact has occurred, a vertical histogram of non-text units is calculated. In the example of FIG. 12, the line histogram 48 should be a uniform value whose height is approximately equal to the line width.
The line width is approximately equal to the width of the text unit (“W”), but if there is a difference, it is due to the slope θs. This tilt occurs when the document image is input. Then, the height 49 of each cell k in the histogram is compared to the width W to determine whether the non-text unit contains a line. The root mean square of the difference between these values is compared to a threshold as:

[0053] This threshold is calculated to allow for twists or slopes of lines in non-text. For 1 ° twist and tilt, Has been found to produce satisfactory results.

If no non-contact line is found by the above inequality, a search is made as to whether or not the line in contact is included. To find out whether the touching line is included in the non-text unit of interest, it suffices to check whether or not a linear object exists near the boundary of the unit. For example, FIG.
If there is a line near the boundary of the rectangle that encloses the unit, as in the example of, can be investigated by calculating the sum of squares of the distances from the boundary. That is, in this case,
The following inequalities are calculated.

[0055] If the left side of the above equation is smaller than a predetermined threshold value, it can be seen that there is a touching line. This threshold may be the same as that for a line that does not make contact.

In this way, if a line is detected in step S28, the process proceeds to step S29, and the attribute (LINE) of "line" is changed to
Appended to the non-text unit. Then, in step S17, the next unit is selected.

On the other hand, if no line is detected in step S28, the flow advances to step S30 to check the size of the non-text unit. If this size is smaller than a certain threshold, the process proceeds to step S31, and the classification of the non-text unit cannot be determined, and the threshold is determined from the maximum font size. Specifically, a good result can be obtained by setting the value to half the maximum font size. Then, an "unknown" attribute (UNKNOWN) is added. After that, the process proceeds to step S17, and the next unit is selected.

In step S30, if the size is larger than a predetermined threshold value, the process proceeds to step S32, in which the white contour is searched for in the internal area of the non-text unit, and the same processing as described in steps S42 to S44 is performed. Processing is step S32 to step S3
4, the number of white contours is calculated.

Then, in step S34, if the number of white contours is not "4" or more, the process proceeds to step S35, and the size is calculated in order to confirm whether or not it has a sufficient size as a line figure or a halftone image. It This size determination is based on the height and width of the unit of text and the maximum run length of black pixels. In particular, if the height and width of the non-text unit are smaller than the maximum font size, it is considered that the unit is not large enough to form a line drawing or a halftone image, and the process proceeds to step S38. An unknown (UNKNOWN) attribute is added.

Similarly, when the unit width is larger than the maximum font size but the maximum length of the black pixel run length is not larger than the maximum font size, the process proceeds to step S38 and the attribute "unknown" is added. And step S1
Returning to 7, a new unit is selected.

In step S35, if the non-text unit has a size large enough to form a line figure or a halftone image, the process proceeds to step S36, it is judged whether it is a line figure, or the line figure or An attribute called a halftone image is added. The processes of these steps S36 to S41 are the same as the above-mentioned steps S55 to S5.
Since the processing is similar to that of 9, the description thereof will be omitted.

5 to 7 (corresponding to step S2 in FIG. 4)
When all the linked components in the input image are examined and classified according to the flow described in the above, the tree structure as shown in FIG. 20 is obtained.

As shown in FIG. 20, the root corresponds to the page of the input image. The root child is a text block
(TEXT) or unknown (NON-TEXT UNKNOWN) frame (FRA
It is a non-text block consisting of ME), photo (picture), and line. The child of a frame (FRAME) is a text block (TEXT), and "unknown" non-text data (UNKNOW).
N), table containing text blocks (TABLE), photo (figure) (PI
CTURE) and line (LINE).

FIG. 23 shows a page 90 of pixel image data, in which a large font size text 91, table 92 containing text data such as 93, text data 94, horizontal lines 95, another. Title 96, text data 97 consisting of two paragraphs,
A line graphic 98 surrounded by a frame with a caption 99, a second column starting with a title 100 and continuing to the text data 101, a halftone image 102 surrounded by a frame including a caption 103, text data 104, a horizontal line 105, and a final line. Paragraph 106 is included.

FIG. 24 shows the same image that has been subjected to the process of step S2 (FIG. 4).

As can be seen from FIG. 24, page 90
The connected component in the inside is cut out in a rectangle, and the inside of the connected component is checked for attributes by the processing shown in steps S23 to S42 to S54 in FIG. Step S11
All the text units obtained in step S12 are grouped vertically or horizontally in step S12 regardless of their position in the tree. This grouping operation is based on the degree of gathering of each text unit and its surrounding units. Also, gaps (spatial spaces) that appear to represent columns are detected and maintained in both vertical and horizontal directions. A detailed description of step S3 in FIG. 4 will be given below with reference to the flowchart in FIG.

First, in step S61, the boundary line of the non-text unit is extended vertically and horizontally to form a gap line marker.

This is as shown in FIG. 24, where the vertical gap line markers 109a, 109b are vertically extended until they intersect a text or non-text unit (in this example, unit 95 of FIG. 23). Similarly, the gap line markers 109c and 109d are also extended until they intersect the unit 95. The same processing is performed for the gap line marker in the horizontal direction.
The gap line marker is effective for detecting a gap (spatial space), whereby a column can be obtained.

Next, the process proceeds to step S62, where 10 in FIG.
Line joining of text units is performed as shown at 7. In the row combination here, the distance between connected components that are close to each other in both directions is checked in advance in each of the horizontal and vertical directions. When the horizontal distance is short, the horizontal connection is performed. When the vertical distance is short, the row combination is performed. This is done in the vertical direction.
The combination direction corresponds to whether the text units to be combined are set vertically or horizontally.

Then, these text units are combined into one text line when the following conditions are met. (1) The bond does not exceed the gap line marker. (2) The text unit is in contact with another text unit or at a distance equal to or less than a certain threshold value. This threshold is obtained by multiplying the average length of the text obtained in step S14 of FIG. 5 by an experimentally obtained scale factor (“1.2” is obtained in this embodiment, which is a satisfactory result). Good.

However, by extending the gap between the text units in the horizontal direction when the text units are in horizontal writing mode and in the vertical direction when the text units are in vertical writing mode before combining, in the direction showing the column structure, It can be determined if there is an extended gap. For example, in the example of FIG. 24, the gap 108 exists between two texts. Since the gap extends in the vertical direction over several lines, in step S62, even if the distance between the text units is equal to or less than the threshold value, the gap is left.

Next, the process proceeds to step S63 and step S6.
For a set of text units that were not joined by two, a join is made when those units are overlapped by another text unit in close proximity and the join does not cross a gap line marker.
This step is effective in erasing anything that does not come from the structure of paragraphs, but simply from the relationship of spaces in a text line. In the example of FIG. 24, the gap 108 left in step S62 is the same as that in step S63.
To be erased. This is because it overlaps the character immediately below and does not cross the gap line marker. Then, in step S64, the tree structure is updated.

FIG. 19 is a schematic diagram showing the result of the grouping process described in step S3 of FIG. 4, and FIG.
It is a figure showing how the tree structure was changed by the process of this step S3.

As shown in FIG. 25, the combined text units are grouped into text lines as shown at 110. In particular, everywhere in the tree structure, text units are always combined into text lines. For example, 111 is below the frame table in a tree structure, but is also joined. However, it should be noted that regrouping beyond the white contour obtained in steps S27 to S39 and steps S42 to S59 of FIG. 6 is not performed. This is because the items in the table are not put into one line. The gap between the left and right columns is maintained. Also, non-text units are not regrouped. Therefore, as indicated by 112 and 113, these units are not grouped even if they are at a distance equal to or less than the threshold.

FIG. 21 shows a tree structure that reflects this new grouping.

After the text units are combined to form a text line in the process described with reference to FIG. 18 (step S3 in FIG. 4), as shown in step S4, the text line is not connected in the direction in which the text line is formed. Combined in the opposite direction to form a text block. This process will be described in more detail with reference to FIG.

The process of grouping depends on the cohesion of the text line units and the positions of the non-text units. For example, intervening non-text lines act as boundaries, and text lines on opposite sides are grouped into one
Prevent it from becoming one text block. All text lines between two consecutive non-text line units are processed simultaneously. In addition, in step S4 some text units should be combined with non-text units (eg text captions composed with non-text images), or some non-text units should be combined with other non-text units. If you have a kika (for example, a line shape associated with a halftone image),
Be examined.

FIG. 19 is a flow chart showing how text lines are grouped into text blocks.

First, in step S71, a title block is formed from those classified as non-text units in step S14. The criterion is that it is smaller than the maximum font size but larger than the average text size. A title block is formed by grouping together all such non-text units of similar size and proximity. Then, an attribute (TITLE) called "title" is added to the block. An attribute of picture text is added to all the remaining non-text blocks that cannot be grouped. The tree structure is then updated accordingly. Note that the title is useful for reconfiguring the page.

Next, in step S72, a non-text unit between text lines is detected. These non-text units act as boundaries between text blocks, preventing text lines from becoming one text block. Then, the process proceeds to step S73, where the text lines are grouped in a direction (hereinafter, referred to as a "block combining direction") opposite to the combining direction at the time of forming the text line by the process of two steps, and the text blocks are grouped. Become. In the first step, gaps between columns are searched. For this purpose, for example, a histogram of the pixel in the block joining direction is calculated. In the second step, if the distance between consecutive text lines in the block combining direction is smaller than the height of the text calculated in step S14 of FIG. 5, these text lines are grouped in each column. The process of step S73 is effective for joining the text lines belonging to the same paragraph as the text line 114 of FIG.

Next, proceeding to step S74, the text blocks which are vertically or horizontally adjacent to each other are not separated by the non-text unit, and step S73
Grouped if it does not break any gaps found in the histogram obtained in. The grouping of these text blocks is performed based on the separation state between blocks smaller than a certain threshold calculated according to the height in the vertical direction calculated in step S14.

In the example of FIG. 25, the processing of step S74 is effective for grouping the text blocks formed from the text lines of paragraph 115 and the text lines of paragraph 116. However, it is not valid to combine paragraphs 117 and 118. These, these text blocks 1
This is because 17,118 are separated by the non-text block 119 (line).

Next, in step S75, it is determined whether a certain text block should be combined with a non-text block or a certain non-text block should be combined with another non-text block. Here, text blocks can be combined with non-text title blocks, non-text halftone blocks, and non-text lines with attachments. These connections are made according to the following judgment.

(1-a) If a text block is close to a non-text title block in the horizontal direction and vertically overlaps, the text block is combined with the non-text title block (however, , Text blocks and title blocks are both in horizontal writing mode).

(1-b) If a text block is vertically close to and horizontally overlaps a non-text title block, the text block is combined with the non-text title block (however, , Text blocks and title blocks are both in vertical writing mode).

(2) If a text block (horizontally and vertically) is smaller than a word-sized block and the text block has no adjacent word-sized text blocks, then this text block is non- Placed inside a text halftone image block.

(3) For a certain text block that overlaps a non-text line with an adjunct, the line with that adjunct is probably underlined text, so it is simply text.

Also, some non-text blocks are
Combined with other non-text blocks according to FIG.
In addition, in FIG. 27, "Test" has the following contents. Test # 1: Combine if one block is completely contained in another block. Test # 2: Combine if the width of the picture text is smaller than the width of the word size block. Test # 3: If the blocks are close to each other, they are combined.

Next, in step S76, the attributes are modified,
The tree structure is updated by the processing described so far.

FIG. 26 shows the block structure obtained by the processing of FIG. 19, and FIG. 22 is a diagram showing an example of the tree structure.

As the blocks in FIG. 26, a title block 120, a text block 121, and a photograph (line drawing)
There is 122. Further, as frame data, a block 123 and a text unit 125 in a tabular form are used.
There is a block 124 having a table structure inside. The non-text image 127 serves as a separator for various units in FIG.

The details of the processing of the attribute separating unit 4 have been described above. The present invention may be applied to a system including a plurality of devices or an apparatus including one device. Further, it goes without saying that the present invention can be applied to the case where it is implemented by supplying a program to a system or an apparatus. In this case, the storage medium storing the program according to the present invention constitutes the present invention. Then, by reading the program from the storage medium to the system or device, the system or device
It works in a predetermined way.

As described above, the classified contents and quantity are sorted based on the attribute separation result of the attribute separating unit 4 and the sorted information is output as a classified information list. By performing the separation processing and collectively printing out the list in one sheet, whether or not to correct the attribute separation can be determined in advance, so that the correction can be efficiently and easily performed.

Further, the attribute separation result can be corrected using the display section 6b integrated with the control panel of the image processing apparatus of the present invention. As described above, it is possible to provide an image processing apparatus having a relatively simple structure and an excellent operating environment.

[0095]

As described above, according to the present invention, the input image can be attribute-separated and its statistical information can be generated to facilitate the subsequent retrieval of the image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of the present embodiment.

FIG. 2 is a diagram illustrating an operation example of a creation unit.

FIG. 3 is a diagram showing an example of a classification information list.

FIG. 4 is a flowchart showing an outline of processing in an attribute separation unit.

5 is a flowchart showing a process of step S2 of FIG.

FIG. 6 is a flowchart showing a process of step S2 of FIG.

FIG. 7 is a flowchart showing a process of step S2 of FIG.

FIG. 8 is a diagram illustrating attribute separation of images included in a document image.

FIG. 9 is a diagram illustrating attribute separation of images included in a document image.

FIG. 10 is a diagram showing a hierarchical structure of rectangular areas in the present embodiment.

FIG. 11 is a diagram illustrating separation of non-text areas.

FIG. 12 is a diagram illustrating separation of non-text areas.

FIG. 13 is a diagram illustrating separation of non-text areas.

FIG. 14 is a diagram showing a search direction inside a non-text unit.

FIG. 15 is a diagram illustrating a case where white contours are combined in the present embodiment.

FIG. 16 is a diagram illustrating a case where white contours are combined in the present embodiment.

FIG. 17 is a diagram illustrating a case where white contours are combined in the present embodiment.

18 is a flowchart showing detailed processing of step S3 of FIG.

FIG. 19 is a flowchart showing how text lines are grouped into text blocks in the present embodiment.

FIG. 20 is a diagram illustrating a tree structure obtained according to the present embodiment.

FIG. 21 is a diagram illustrating a tree structure obtained according to this embodiment.

FIG. 22 is a diagram illustrating a tree structure obtained according to this embodiment.

FIG. 23 is a diagram showing a specific example of an original image processed in the present embodiment.

FIG. 24 is a diagram showing an example in which the image of FIG. 23 is divided into units.

FIG. 25: Step S2 for the same image (FIG. 4)
It is a figure which shows the example which performed the process of.

FIG. 26 is a diagram showing a block structure obtained by the processing of FIG.

FIG. 27 is a diagram showing logic for connecting non-text blocks to each other.

[Explanation of symbols]

 1 Scanner Section 2 Memory Control Section 3 Memory Section 4 Attribute Separation Section 5 Creation Section 6 Output Section

Claims (21)

[Claims] 1. An image input step of inputting an image, an attribute separation step of separating the image input in the image input step into partial images of a predetermined attribute, and the same attribute generated in the attribute separation step. An image processing method comprising: a statistical processing step of obtaining statistics of partial images; and an image forming step of forming an image of statistical results of the partial images generated in the statistical processing step. 2. The image processing method according to claim 1, wherein the image forming step forms an image on a display unit. 3. The image processing method according to claim 1, wherein the image forming step forms an image on an image recording apparatus. 4. The image processing method according to claim 1, wherein the statistic is the number of partial images having the same attribute. 5. The image processing method according to claim 1, wherein the predetermined attribute includes a title part. 6. The image processing method according to claim 1, wherein the predetermined attribute includes a table portion. 7. The image processing method according to claim 1, wherein the predetermined attribute includes a graph portion. 8. The image processing method according to claim 1, wherein the predetermined attribute includes a text portion and a non-text portion. 9. The image processing method according to claim 1, wherein in the attribute separating step, image elements in which the distance between image elements in the input image is smaller than a predetermined distance are included in the same group. . 10. The attribute separating step determines whether the image attribute is text or non-text according to the number of connected white pixels or black pixels in the input image. The image processing method described in. 11. An image input unit for inputting an image, an attribute separating unit for separating an image input by the image input unit into partial images having a predetermined attribute, and an attribute separating unit of the same attribute generated by the attribute separating unit. An image processing apparatus comprising: statistical processing means for obtaining statistics of partial images; and image forming means for forming an image of statistical results of the partial images generated by the statistical processing means. 12. The image processing apparatus according to claim 11, wherein the image forming unit forms an image on a display unit. 13. The image processing apparatus according to claim 11, wherein the image forming unit forms an image on an image recording apparatus. 14. The image processing apparatus according to claim 11, wherein the statistic is the number of partial images having the same attribute. 15. The image processing apparatus according to claim 11, wherein the predetermined attribute includes a title portion. 16. The image processing apparatus according to claim 11, wherein the predetermined attribute includes a table portion. 17. The image processing apparatus according to claim 11, wherein the predetermined attribute includes a graph section. 18. The image processing apparatus according to claim 11, wherein the predetermined attribute includes a text portion and a non-text portion. 19. The attribute separating means sets the image elements in which the distance between the image elements in the input image is smaller than a predetermined distance into the same group.
An image processing apparatus according to claim 1. 20. The attribute separating means determines whether the image attribute is text or non-text according to the number of connected white pixels or black pixels in the input image. The image processing device according to item 1. 21. The image processing apparatus according to claim 11, wherein the image input unit is a scanner that inputs light reflected from an image and converts the light into a corresponding electric signal.