JPH10320557A - Method and apparatus for detecting telop character display frame - Google Patents

Abstract

(57) [Summary] (With correction) [Problem] To reduce the rate of erroneous detection and improve the telop character display frame detection accuracy. SOLUTION: A telop character display frame candidate on which a predetermined telop character is displayed is detected, and only a plurality of predetermined sub-parts which are not allowed to overlap with each other only partial blocks satisfying the condition of the telop character display candidate. The image is divided into block areas, edges are detected for each sub-partial block, and the edge detection result image and the original grayscale image are scanned in a preset direction. Is determined, and two adjacent lines on a certain scan line in a certain scan direction are determined.
If the gradient directions of the two edges are opposite, they are counted as one set of edge pairs, the number of edge pairs is counted for all preset scan directions, and the ratio of the number of edge pairs found for all scan directions is calculated. It is calculated, and it is determined whether or not the ratio of the edge pair in each scanning direction falls within a preset range.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for detecting a telop character display frame, and more particularly to a technique for detecting a frame in which a character is displayed from a plurality of frames constituting a moving image. Technology.

[0002]

2. Description of the Related Art With respect to a technique for detecting a frame including a character from a plurality of frames constituting a moving image,
In recent years, many studies have been made, and among them, many methods based on a luminance difference between frames reflecting a difference between character display and non-display are proposed.

[0003] Among them, in addition to the luminance difference between frames, a feature amount based on the luminance distribution in the frame is calculated,
Kuwano, Kurakake, and Odaka have proposed a method for detecting frames containing characters. "A telop character extraction method for video data search" (IEICE Technical Report, PRMU96-98,
pp39-46, (1996)). This is because each frame is divided into a plurality of partial rectangular areas, and the luminance histogram difference between each partial rectangular area of each frame and the corresponding partial rectangular area of the frame at the previous time is larger than the set threshold. The value of the area of the high-contrast portion that is large and does not change its existing position continuously with the corresponding partial rectangular region of the frame at the later time is obtained. As an appearance frame candidate, next, edge detection with direction information is performed in this frame, and within a preset local range in the frame,
If the number of edges and the directional variance of the edges are larger than a preset threshold, the frame is set as a character appearance frame.

[0004]

However, in the above-mentioned conventional method, objects other than telop characters (for example, a checkerboard pattern) having a large number of high-contrast parts and having a high number of edge points and a high edge variance are obtained. (Clothes with different colors of clothes worn by each person), etc., and then appear stationary for a certain period of time, the frame when these objects appear is erroneously detected as a telop character display frame was there.

It is an object of the present invention to provide a technique capable of reducing a failure in detecting a telop character display frame due to instability of image characteristics and an excessive detection.

Another object of the present invention is to provide a technique capable of reducing the rate of erroneous detection and improving the accuracy of detecting a telop character display frame.

[0007] The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0008]

In order to achieve the above object, according to the present invention, in a character display frame detecting method, a moving image is input in units of frames, and the input frames are set in advance without overlapping each other. Divided into a plurality of set partial blocks, and in each frame divided into the partial blocks, a telop character display frame candidate displaying a predetermined telop character is detected for each partial block. In the divided telop character display frame candidate, only the partial blocks that satisfy the condition of the telop character display candidate are divided into a plurality of preset sub-sub-block regions that do not allow overlapping each other. To perform edge detection,
The edge detection result image and the original grayscale image are scanned in a preset direction, the gradient direction of each edge existing on each scan line in the scan direction is determined, and the edge direction is determined on a certain scan line in a certain scan direction. If the gradient directions of the two edges are opposite, they are counted as one set of edge pairs, the number of edge pairs is counted for every preset scan direction, and the edge pairs found for all preset scan directions are counted. Is calculated, it is determined whether or not the ratio of edge pairs for each scanning direction falls within a preset range.If it is determined that the ratio of edge pairs for each scanning direction falls within a preset range, The subsequent processing is not performed on unprocessed sub-partial blocks and partial blocks in the processing target frame, and characters are displayed on the processing target frame. It determines that the frame, is characterized in that writing to the storage medium. The moving image is input in frame units and stored in a memory.

In the character display frame detecting device, means for inputting and storing a moving image in frame units;
Means for dividing each frame into a plurality of predetermined partial blocks that do not allow overlapping, and a telop in which a predetermined telop character is displayed for each partial block in each of the divided frames. Means for detecting a character display frame candidate, and dividing the detected telop character display frame candidates into only a plurality of preset sub-sub-block regions that do not allow overlapping with each other only the partial blocks that satisfy the telop character display candidate conditions. Means for performing edge detection processing using a predetermined edge detection method for each sub-partial block, and scanning the edge detection result image and the original grayscale image in a predetermined direction, and Means for determining the direction of their gradient for each edge present on each scan line of If the two gradient directions of edges adjacent scan lines with that the scanning direction is the opposite direction,
Means for counting them as one set of edge pairs and counting the number of edge pairs for all preset scan directions;
Means for calculating the ratio of the number of edge pairs determined for all scan directions set in advance; means for determining whether the ratio of edge pairs for each scan direction falls within a preset range; and edge pairs for each scan direction. If it is determined that the ratio falls within the preset range, the subsequent processing is not performed on the unprocessed sub-partial blocks and the partial blocks in the processing target frame, and characters are displayed on the processing target frame. And a means for judging a frame to be written to the storage medium.

[0010] The color inside the telop character in the video is a single color, and the color relationship with the character surrounding portion is characterized by high contrast. It has the characteristic that the color around the telop character is also a single color. When an edge is detected in a local region in an image including a telop character, the local region is scanned from various directions, and a gradient direction of an adjacent edge on a certain scanning line is checked, an edge obtained from a telop character outline is obtained. Has a relationship of “up” and “down” or “down” and “up” from the telop character and the surrounding color features. (Two adjacent edges having this relationship are defined as "one edge pair.") Since a character has strokes having various directions, an approximately equal number of edge pairs can be obtained in all scan directions. Has nature.

On the other hand, with respect to an object other than a telop character which includes many high-contrast portions, and is characterized in that the number of obtained edge points and the direction variance of the edge are high, the color inside the object usually has various variations. And the relationship with the color of the surrounding area of the object is not always high when the lighting conditions are also taken into account. From this, edge detection is performed in the local area including this object,
Even if the local region is scanned from various directions, the number of the edge pairs described above may not be uniformly obtained from each scanning direction.

Therefore, the above-described edge detection result image and the original grayscale image are scanned in a predetermined direction, and for each edge existing on each scanning line in the scanning direction, the direction of their gradient is determined, and a certain scanning direction is determined. When the gradient directions of two adjacent edges on a certain scanning line are in opposite directions, they are counted as one set of edge pairs, the number of edge pairs is counted for every preset scan direction, and all the preset scans are performed. The ratio of the number of edge pairs obtained for each direction is calculated, and it is determined whether the ratio of edge pairs for each scan direction falls within a preset range. The ratio of edge pairs for each scan direction falls within a preset range. Is determined, the subsequent processing is not performed on the unprocessed sub-partial blocks and partial blocks in the processing target frame, and the processing target By judging the frame as a frame displaying characters and writing it to the storage medium, the frame at the time of appearance of an object having a high-contrast part other than the telop characters, which was a problem of the conventional method, is incorrectly regarded as a telop character display frame. Detection can be suppressed.

[0013]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of an embodiment (embodiment) of the present invention. 1 is a moving image input storage means for inputting moving image data to be processed and storing it in a memory; It is a telop character display frame candidate detecting means, and detects a frame candidate displaying a telop character from moving image data input and stored in the moving image input storage means 1 by using an existing method.

Reference numeral 3 denotes a telop character display frame candidate judging means. The telop character display frame candidate detected by the telop character display frame candidate detecting means 2 is used for telop character display frame candidates by using a feature amount obtained from the luminance distribution in the frame. Determine if characters are being displayed.

Reference numeral 4 denotes a telop character display frame storage / storage means for outputting and storing a frame determined by the telop character display frame candidate determining means 3 to be a telop character display frame.

FIG. 2 is a block diagram showing a detailed configuration of one embodiment of the telop character display frame candidate judging means of the present embodiment. In FIG. 2, reference numeral 31 denotes an unprocessed partial block determining unit which determines a telop character display frame candidate for a partial block of the telop character display frame candidates detected by the telop character display frame candidate detecting unit 2 in FIG. If there is an unprocessed one, the processing is transferred to the sub-partial block dividing means. If all the blocks have been subjected to the telop character display frame candidate determination processing, the telop character display frame candidate detecting means 2 of FIG. Control is performed to shift the processing to the frame candidate detecting means.

Reference numeral 32 denotes a sub-partial block dividing means for dividing the partial block into a plurality of sub-partial blocks which are not allowed to overlap each other, 33 denotes an edge detecting means for performing an edge detecting process in the sub-partial block, and 34 denotes a scan control means. If there is an unprocessed scan direction of the edge pair processing for the sub-partial block subjected to the edge detection processing, the processing is transferred to the scan control unit 34, and if the edge pair processing is completed for all the scan directions Control to transfer the process to the edge pair account means is performed.

Reference numeral 35 denotes edge gradient direction judging means for judging the gradient direction of an edge along a certain scanning direction. Reference numeral 36 denotes edge pairing means, which is “up”, “down”, “down”, or “down” along the scanning direction. The number of two adjacent edges (edge pairs) is counted in the order of “climbing”.

Reference numeral 37 denotes direction edge pair ratio calculating means for calculating the ratio of edge pairs counted for each scanning direction in each scanning direction, and reference numeral 38 denotes direction edge pair ratio calculating means 3.
It is determined whether the edge pair ratio for each direction calculated in step 7 belongs to a preset range, and if it does, the processing is shifted to the telop character display frame accumulation storage means 4 in FIG. Direction-based edge pair ratio determination means for performing control to shift processing to the block determination means 31.

FIG. 3 is a flowchart for explaining the telop character display frame detection processing of this embodiment.

In FIG. 3, a step (31) is a step of judging whether or not there is an unprocessed frame among the frames constituting the moving image to be subjected to the telop character display frame detection processing. Then, the process proceeds to step (32), and if there is no unprocessed frame, the process ends.

Step (32) is a process of reading a frame sequence including a frame to be subjected to the telop character display frame detection processing and other frames necessary for the processing into the memory.

Step (33) is a process of dividing each input frame into a plurality of partial blocks which are not allowed to overlap each other. FIG. 4 is a diagram for explaining an embodiment of the partial block division in step (33), in which the frame is divided into two vertically and four horizontally and divided into a total of eight partial blocks.

In the step (34), a conventional technique, for example, a "telop character extraction method for video data retrieval" is applied to each partial block of the frame to be processed (IEICE Technical Report, PRM
U96-98, pp39-46, (1996)).

Step (35) is a process of reading the telop character display frame candidates detected in step (34) into the memory.

Step (36) is a process of judging whether or not there is an unprocessed partial block in the telop character display frame candidate determination process. If there is an unprocessed partial block, the process proceeds to step (37). If the partial block does not exist, the process proceeds to step (31).

Step (37) is a process of dividing the partial block into a plurality of non-overlapping sub-partial blocks. FIG. 5 is a diagram showing an example of sub-sub-block division in step (37), in which the sub-block is divided into seven vertically and four horizontally and divided into a total of 28 sub-sub blocks.

Step (38) is a step of judging whether or not there is any unprocessed sub-partial block in the edge pair processing. If there is an unprocessed sub-partial block, the process proceeds to step (39), and the process proceeds to step (39). If there is no sub-partial block, the process proceeds to step (36).

Step (39) is a step of performing edge detection with gradient direction information in the sub-partial block.
For the edge detection accompanied by the gradient direction information, for example, a Robinson edge detection operator may be used.

Step (40) is a step of judging whether or not an unprocessed scan direction of the edge pair processing exists. If an unprocessed scan direction exists, the process proceeds to step (41), and the unprocessed scan direction is performed. If does not exist, the process moves to step (44). FIG. 6 shows the step (40).
FIG. 6 is an example in which the scanning direction is set to two directions, a horizontal direction and a vertical direction, and the direction of the arrow indicates the scanning direction.

Step (41) is a process of judging whether there is an unprocessed edge point in the process of judging the gradient direction in the scanning direction. If there is an unprocessed edge point, the process proceeds to step (42). If there is no unprocessed edge point, the process proceeds to step (40).

In step (42), it is determined whether the gradient direction of the edge existing in the scan direction is "up" or "down", and the determined direction is the same as the gradient direction of the adjacent edge for which the determination of the gradient direction has already been completed. This is a process of determining whether the direction is the direction or the opposite direction. If the direction is the same, the process proceeds to step (41). If the direction is the opposite, the process proceeds to step (43). FIG.
Is a diagram showing an example of step (42), wherein a grayscale image;
5 is a graph in which the horizontal axis represents coordinate values on a certain horizontal scanning line on a grayscale image, and the vertical axis represents the image density level at the coordinates. Regarding the gradient direction of the edge, “up” means a change from low density to high density, and “down” means a change from high density to low density.

Step (43) is a process in which the number of edge pairs is incremented by 1 when the gradients of two edges adjacent in the scanning direction are in opposite directions ("up", "down", or "down", "up"). is there. FIG. 7 is a diagram for explaining the step (43), in the case where the gradients of two adjacent edges on a certain scanning line have the same direction of “up”, “up”, or “down” or “down”. Does not count edge pairs, and in the case of “up”, “down”, or “down” or “up”, when the edge pairs are incremented by 1, the number of edge pairs on the scanning line shown in FIG. 7 is 13.

Step (44) is a process of calculating the ratio of the number of edge pairs determined for each scanning direction and determining whether or not the ratio belongs to a preset range. Go to 45), otherwise go to step (38).

FIG. 8 is a diagram for explaining the step (44). When the scanning direction of the sub-partial block is set to the horizontal direction and the vertical direction, the total of the edge pairs on all the scanning lines in the horizontal direction is 25, and the vertical The total of the edge pairs on all the scanning lines in the direction is 12, and the ratio of the edge pairs obtained for each scanning direction is horizontal direction: vertical direction = 25: 12. Note that the number of edge pairs in FIG. 8 indicates a point at which a gray portion changes from a gray portion to a white portion on a scanning line, the “climbing” edge,
The point at which the white part changes to the gray part is the “down” edge, and is a number counted based on step (43).

In step (45), a frame including partial blocks including sub-partial blocks determined in step (44) as having a ratio of edge pairs for each scanning direction belonging to a preset range is stored as a telop character display frame. This is the process of writing to.

As described above, the invention made by the present inventor is:
Although specifically described based on the embodiment (example),
The present invention is not limited to the above-described embodiment (example), and it is needless to say that various changes can be made without departing from the gist of the present invention.

[0039]

As described above, according to the present invention, after an object other than a telop character appears in a frame,
In the case where the image is displayed stationary for a certain period of time and contains many high-contrast parts, the number of obtained edge points and the characteristic that the directional variance value of the edge is high, the gradient in the opposite direction for each scanning direction is set. By calculating adjacent edge pairs and calculating their ratio, it is possible to suppress erroneous detection of the frame at the moment when the object appears as a telop character display frame. Accordingly, it is possible to detect the telop character display frame with high accuracy with a lower erroneous detection rate than the conventional method.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an embodiment (executing apparatus) of the present invention.

FIG. 2 is a block diagram illustrating a detailed configuration of an example of a telop character display frame candidate determining unit according to the embodiment;

FIG. 3 is a flowchart illustrating a telop character display frame detection process according to the embodiment;

FIG. 4 is a diagram for supplementing the processing content of step (33) in FIG. 3, and showing an example in which one frame is divided into eight partial blocks.

FIG. 5 is a diagram for supplementing the processing content of step (37) in FIG. 3 and showing an example in which one partial block is divided into 28 sub-partial blocks.

6 is a diagram for supplementing the processing content of step (40) in FIG. 3 and showing an example of a scan direction.

FIG. 7 is a diagram for supplementing the processing contents of steps (42) and (43) in FIG. 3, and showing an example of edge gradient determination and edge pair counting.

8 is a diagram for supplementing the processing content of step (44) in FIG. 3 and showing an example of calculating an edge pair ratio for each scanning direction.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Moving image input storage means, 2 ... Telop character display frame candidate detection means, 3 ... Telop character display frame candidate judgment means, 4 ... Telop character display frame accumulation storage means,
31: unprocessed partial block determining means, 32: sub-partial block dividing means, 33: edge detecting means, 34: scan controlling means, 35: edge gradient direction determining means, 36: edge pairing means, 37: edge pair ratio by direction Calculating means, 38... Direction-based edge pair ratio determining means.

Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI // H04N 5/445

Claims (3)

[Claims] 1. A moving image is input in units of frames, and each of the input frames is divided into a plurality of preset partial blocks that are not allowed to overlap with each other. In each of the divided frames, For each partial block, a telop character display frame candidate displaying a predetermined telop character is detected, and only the partial blocks that satisfy the telop character display candidate condition in the detected telop character display frame candidate overlap each other. Is divided into a plurality of pre-set sub-sub-block areas, and edge detection is performed for each of the sub-sub-blocks. The edge detection result image and the original grayscale image are scanned in a predetermined direction. Then, for each edge present on each scan line in the scan direction, the direction of their gradient is determined, and When the gradient directions of two adjacent edges on a certain scan line in the scan direction are opposite, they are counted as one set of edge pairs, the number of edge pairs is counted for every preset scan direction, and the number of edge pairs is set in advance. Calculate the ratio of the number of edge pairs obtained for all scan directions, determine whether the ratio of edge pairs for each scan direction falls within a preset range, and set the ratio of edge pairs for each scan direction to a preset range. If it is determined that the character is displayed, the subsequent processing is not performed on the unprocessed sub-partial blocks and partial blocks in the processing target frame, and the processing target frame is determined to be a frame in which characters are displayed, and stored. A character display frame detection method characterized by writing on a medium. 2. The character display frame detecting method according to claim 1, wherein the moving image is input in frame units and stored in a memory. 3. A means for inputting and storing a moving image in frame units, a means for dividing each frame into a plurality of preset partial blocks that do not allow overlapping each other, and a means for dividing each frame into partial blocks. Means for detecting, for each partial block, a telop character display frame candidate on which a predetermined telop character is displayed; and a partial block satisfying the telop character display candidate condition in the detected telop character display frame candidate. Means for dividing the sub-blocks into a plurality of predetermined sub-sub-block areas that do not allow overlapping with each other, means for performing an edge detection process on each of the sub-sub-blocks using a predetermined edge detection method, The detection result image and the original grayscale image are scanned in a preset direction, and exist on each scanning line in the scanning direction. For each edge, means for determining the direction of the gradient, and when the gradient directions of two adjacent edges on a certain scan line in a certain scan direction are opposite directions, they are counted as one edge pair and set in advance. Means for counting the number of edge pairs for all scan directions, means for calculating the ratio of the number of edge pairs determined for all scan directions set in advance, and the ratio of edge pairs for each scan direction within a preset range Means for determining whether or not to enter, and when it is determined that the ratio of the edge pair for each scanning direction falls within a preset range, the unprocessed sub-partial block in the processing target frame, and the subsequent , A frame to be processed is determined to be a frame in which characters are displayed, and a means for writing to a storage medium is provided. Display frame detection device.