JPH0432960A - Reading processor - Google Patents

Abstract

PURPOSE:To improve the reading environment of the disabled by providing a means which selects and reads the document of a color mark part of a document or book freely. CONSTITUTION:An image input part 1 outputs information on the density or/and colors of an image of characters, a figure, a photograph and/or mark, etc., entered into a form. Then this processor is equipped with an extraction mode switching part 23 which outputs color characteristics of reading information as read color characteristics according to the name of a reading area specified by a reading mode specification part 223 as an image processed part and an image extraction part 24 which extracts a document image in a mark area entered into the document or book according to the read characteristics. For the purpose, color marks are added or printed at desired parts of the document or book and the document of the color mark parts of the document or book or all sentences can be selected freely by voice guidance or voice recognition. Consequently, the reading environment of the disabled such as the blind and the physically handicapped can be improved.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a reading processing device that recognizes characters on a document or book and outputs a reading voice based on the recognition result.

[Prior art]

Conventionally, this type of device was published in IEICE Technical Report Vol. 1.87 No. 334 1988
Published on January 22, 2016, there was something disclosed on pages 79-86. This device has a configuration as shown in Fig. 2, in which an image input section 1 inputs a document or book page by page, and an image processing section 2 performs image processing such as tilt correction on the input image. , the document image characters outputted from the image processing section 2 are cut out and recognized by the character recognition processing section 3, then subjected to language processing by the language processing section 4, converted into speech by the speech synthesis section 5, and then processed by the speech output section 6. It is designed to output audio from.

[Problem to be solved by the invention]

However, with the conventional reading processing device having the above-mentioned configuration, it has been impossible for healthy people to select the reading method described below, which they do on a daily basis when reading documents and books. For example, if the document or book to be read is an essay, (1) Read the bibliographic information such as 11 names and authors (2) Prove the summary (3) Read the title of chapter 2, etc. (4) Read the references (5) After knowing the above, decide whether to read the entire text or not.

In addition, it could not be said that it was a good reading environment for people with disabilities such as blind people and people with physical disabilities.

The present invention has been made in view of the above-mentioned problems, and in order to eliminate the above-mentioned problems, it is possible to freely select and read the text in the color mark part of documents and books, so that sighted people can read it on a daily basis. To provide a reading processing device capable of reading in a manner similar to the browsing method of reading documents and books, which is useful for improving the reading environment for handicapped people.

[Means to solve the problem]

In order to solve the above problems, the present invention provides a reading processing device that includes an image input section, an image processing section, a character recognition section, a language processing section, and a speech synthesis section, and outputs a reading voice based on the result of character recognition. As an image processing section, an image input section is used that outputs information corresponding to one or both of the shading and color of an image such as a character 0 figure, photograph, and/or mark written on a sheet of paper.As an image processing section, a predetermined a color command setting section that sets a color characteristic and a reading area name as a pair of reading information;
a reading mode specifying section that specifies a reading area name through voice interaction processing; an extraction mode switching section that outputs a color characteristic of reading information as a reading color characteristic based on the reading area name specified by the reading mode specifying section; Read documents based on color characteristics.

The present invention is characterized by using an image processing section that includes an image extraction section that extracts a document image within a mark area written on a book.

[Effect]

Since the reading processing device of the present invention has an image input unit and an image processing unit as described above, it adds or prints a color mark to a desired part of a document or book, and prints the text in the color mark part of the document or book. Alternatively, it becomes a reading processing device that can freely select the entire text using voice guidance and voice recognition. Therefore, if you use this book reading processing device, you can print documents or books that have been printed by a bookbinder or once added by an able-bodied person, and which are printed on a daily basis by a able-bodied person.

Since it is possible to read in a manner similar to browsing a book, it is useful for improving the reading environment for people with disabilities such as blind people and physically disabled people, and it is also possible to provide an auditory reading environment for healthy people.

〔Example〕

Hereinafter, one embodiment of the present invention will be described based on the drawings.

FIG. 1 is a block diagram showing the configuration of a reading processing device that is an embodiment of the present invention. As shown in the figure, the reading processing device includes an image input section 1, an image processing section 2, a character recognition section 3, a language processing section 4, a speech synthesis section 5, and a speech output section 6.

The image processing section 2 includes a color command setting section 21, a reading mode specifying section 22, an extraction mode switching section 23, and an image extraction section 24.
It consists of

Further, the color command setting section 21 includes a learning/reading mode switching section 211 and a color command learning section 212,
The reading mode specifying section 22 includes a voice input section 221, a voice recognition section 222, and a reading mode setting section 223.

The image input unit 1 scans a sheet of paper as shown in FIGS. 4 and 5 with main scanning in the horizontal direction (XX) and sub-scanning in the vertical direction (YY), and records marks and/or marks written on the paper. or Document 9 Color information corresponding to one or both of the shading and color of the drawing, such as a density multilevel signal or R (red), G (green),
A B (blue) multilevel signal is output to the color command setting section 21 as color information.

Learning/reading mode switching section 2 of color command setting section 21
Reference numeral 11 denotes a switching unit that switches between a learning mode and a reading mode using a switch (not shown) or the like, and when the learning/reading mode switching unit 211 is in the learning mode, the output of the image input unit 1 is switched to the color command learning mode. 212, and also outputs a learning start signal. Furthermore, when the mode is switched to reading mode, the output of the image input section 1 is outputted to the image extraction section 24, and an image extraction activation signal is outputted.

The color command learning unit 212 learns the color characteristics of a predetermined area of the color command study paper as shown in FIG. A character code string corresponding to a reading area name determined from a blank character string and the color characteristics are set as a pair of reading information.

The color command setting section 21 described above is configured to set reading information by learning the color characteristics and reading area name on the color command learning paper output from the image input section 1, but the color command setting section 21 is configured to set reading information by learning the color characteristics and reading area name on the color command learning paper output from the image input section 1. Instead of using color information, reading information may be set by selecting from predetermined color characteristics and reading area names.

As mentioned above, the reading mode designation section 22 is connected to the voice input section 22.
1 and a voice recognition unit 222, and outputs a character string corresponding to a desired reading area name through voice interaction processing. The voice input unit 221 converts the reading area name, voice for voice dialogue processing, etc. uttered by the person who wishes to read into voice signals. The voice recognition unit 222 recognizes the voice signal and outputs a character code string.

Note that the speech recognition unit 222 retains speech recognition parameters in order to accurately recognize reading area names and words used in speech dialogue processing.

The reading mode setting unit 223 performs the following processing and control in order to set the reading area name by voice interaction processing.

■Comparing the character code string output from the speech recognition unit 222 with the character code string for dialogue processing, ■Outputting a character food string for voice guidance to the speech synthesis unit 5 based on the matching result of (■) above, ■ Matching the character code string output from the voice recognition unit 222 with the character code string corresponding to the reading area name set in the color command setting unit 21, ■ Based on the matching result of above ■, reading area name Outputs the corresponding character code string.

The extraction mode switching unit 23 outputs the color characteristics of the reading information to the image extraction unit 24 as reading color characteristics based on the character code string corresponding to the reading area name output from the reading mode designation unit 22.

The image extraction unit 24 extracts a document image within a mark area written on a document or book as a reading area image based on the reading color characteristics.

The extracted reading area image is subjected to processes such as tilt correction as in the prior art, and character images are cut out character by character.

The character recognition processing unit 3 sequentially recognizes the cut out character images,
Create a character code string.

The speech synthesis section 5 converts the character code string and the character code string for voice guidance outputted from the reading mode setting section into an audio waveform signal.

The audio output unit 6 converts the audio waveform signal from the audio synthesis unit 5 into audio and outputs it as reading audio and guidance audio.

Fig. 3 is an explanatory diagram of color characteristics showing the principle for classifying marks on documents and books by color, and Fig. 3 (a) is a chromaticity diagram showing the range of color coordinates of marks; (b) is an explanatory diagram of the density range of the mark.

When using the principle shown in FIG. 3(a), the image input section 1 requires a color scanner capable of outputting, for example, an RGB density multilevel signal.

When using the principle shown in FIG. 3(b), the image input section 1 requires a monochrome scanner or the above-mentioned color scanner capable of outputting a multilevel density signal.

The chromaticity diagram in FIG. 3(a) is a schematic diagram created by a known method, in which the RGB system density signal is converted to the XYZ system using coordinate conversion formula (1), X −2,7689R+ 1.7517 G + 1.
1302 BY = R+ 4.5907 G
+ 0.0601 BZ= 0.056
5G+5.5943B (1) Obtained,
Calculate the chromaticity coordinates x and y using formula (2) based on Y and Z, and calculate the chromaticity coordinates x and y as follows: It uses orthogonal coordinates. The X and y represent hue and saturation, and Y represents lightness.

Conversion C from the above RGB system to the XYZ system <Use (1)] Calculation to obtain x and y representing hue and saturation [Equation (2)
], input R, G, B, x, y, z
using a ROM that outputs and inputs X, Y, Z,
By converting x t 3' using a ROM that outputs it in an integer format, no calculation time is required and processing speed can be increased.

In the case of Fig. 3(b), the document or book to be extracted is white;
It is expressed in black, and the density level of the mark on the document or book is a density level that does not overlap with the density of the white or black, and the color of the mark has a density level that does not overlap with each other for each color. Color classification becomes available by selecting a color. Moreover, the above Y may be expressed similarly.

FIG. 4 is a diagram showing an example of a color command learning sheet, in which (a) is a character entry frame m, 1. and a color command entry frame m3+ for entering a color mark corresponding to the reading area name, which are vertically arranged.

In addition, Fig. 4(b) shows a vertical color command that integrates a character entry frame m* for entering the reading area name and a color command entry frame for entering the color mark corresponding to the reading area name. They are arranged in the direction.

Further, in FIG. 4(C), the reading area name is uniquely determined by the position of the color command entry frame mH.

In FIG. 4, it is assumed that the entry frame is printed in black.

Note that the color, size, shape, number, and positional relationship of the entry frames are not limited to those shown in FIG. 4, and the reading area name may be an abbreviation.

In Figure 4, the mark g1 written on the color command learning sheet is red, g! is green, g is blue, and g4 is yellow.

FIG. 3(a) schematically shows the position of the mark on the chromaticity diagram.

Figure 5 is an example of an input document in which an able-bodied person has added additional marks in the desired reading area of a paper-style document, where m is a red (g) color mark added in the area of bibliographic items; m,
is the green (g,) color mark added in the summary area, m
st, m,, are added in blue (
g,), which have been added to correspond to the contents filled in on the color command study form in Fig. 4.

Next, the processing of the color command learning section 212 will be explained. The image input unit 1 scans the color command learning sheet shown in FIG. 4 with the main scanning in the horizontal direction (XX) and the sub-scanning direction (YY), and the scanning point is in the color command entry frame m, 1 or in the color command entry frame in FIG. 4 (b). ) character entry frame m, 1. If the output of the image input unit 1 is determined to be a chromatic color within the area of
The R, G, and H multi-level signals output from the image input unit 1 are converted into color coordinates x, y using the above equations (1) and (2).
Then, the reading area name number is written on the color map using X% and y%, which are obtained by converting X+3' into integers, as addresses. As shown in FIGS. 4(a) and 4(b), if the reading area name number has not been determined, the number assigned in the position order of the color command entry frame, for example, in ascending order of YY coordinates, is used.

The aforementioned chromatic color determination is performed by a known method, for example, by making the signal levels of R2O and H different.

Next, when learning reading area names as shown in the color command learning sheets shown in Figures 4(a) and (b), the image within the character entry frame is extracted and a known character recognition method is used to extract the image within the character entry frame. The recognized character string is used as a reading area name, and the reading area name number and reading area name assigned to the color command entry frame corresponding to the character entry frame, and the above-mentioned color map are used as reading information. Color command setting section 2
1 in the color command learning section 212.

Next, a process for operating the reading mode specifying section 22 and the present reading processing device will be described using FIG. 6.

In FIG. 6, the part shown in hiragana in parentheses is the voice command to be voice recognized by the person who wishes to read, and indicates the words for dialogue processing and the name of the reading area. ′
The part indicated in katakana in "" is the character code string for interaction processing registered in advance in the reading mode setting section 223 of the reading mode specifying section 22 and the color command setting section 21.
The character code string corresponding to the reading area name set in the color command learning section 212 indicates the synthesized speech outputted via the speech synthesis section 5 and the speech output section 6. First, to request voice guidance to know what kind of reading mode is possible, input the voice "1 area" from the voice input section 221, and when the voice recognition section 222 recognizes it as "Ready to go" (step STI, 5T2), the character code string corresponding to the reading area name that has been set in the color command learning section 212 of the color command setting section 21, such as "Bibliographical matters", 'Summary', the title of Chapter 1, Section 2, etc. ..."Full text" etc. is output.

Next, in order to specify the reading mode, when the voice recognition unit 222 recognizes ``Bibliographical matters'' and ``Let's go ahead'', the parameters registered in the reading mode learning process described above are used. The voice is recognized by the voice recognition unit 222, and if it is recognized correctly (step 5T4), the voice corresponding to the character code string of the recognition result is echoed back (step 5T11), and then a check is made as to whether or not it is in the specified reading mode. Synthesized speech is output for confirmation (step 12).

If the specified reading mode is the desired mode, "
When the voice “OK” is input (step 5T13), the mode switching unit 2 extracts the reading area name based on the voice recognition result.
3 (step 5T14). When specifying other reading modes, follow the same procedure as described above.

In step 13, if a voice other than ``10K'' is input if the reading mode is not the desired one, a synthesized voice saying ``More'' is output (step 5T8), and the process waits for the next voice command to be input (steps 5T13, 8. 1).

When the designation of the reading mode is completed, ``1 start'' is uttered to start reading, and when the voice recognition unit 222 recognizes ``1'' (step STI).

2.3) In order to start the reading operation, the image input unit 1 starts scanning the set document or book.

The extraction mode switching unit 23 selects the reading area name of the reading information registered in the color command learning unit 212 of the color command setting unit 21 based on the reading area name output from the reading mode setting unit 223 of the reading mode specifying unit 22. , and mask the color characteristics other than the reading area name number (0
The resulting color map is output to the image extraction section 24. In the case of the preamble reading mode, a full text reading signal is output to the image extracting section 24 without outputting a color map.

Next, the mark identification process and the image extraction process in the mark area by the image extraction section 24 will be described.

Mark identification processing is performed as follows. Image input section 1
scans a document like the one shown in Figure 5 in the horizontal direction (XX)
, the RGB multi-level signal outputted by scanning with the sub-scanning in the vertical direction (YY) is converted into chromaticity coordinates x, y using the above equations (1) and (2), and the x, y X, which is an integer
%, y% to refer to the color map output from the extraction mode switching section 23, and if the output value of the color map is other than "rO", the scanning point is on the mark indicating the desired reading area. identify.

The image extraction process within the mark area will be explained using FIG. 7.

In Fig. 7, no mark image is detected in the sub-scanning YYI row, but the color characteristics (reading area name number) are detected in the YYZ row.
The mark image of j is identified by the mark identification process described above, and furthermore, in the YYS row, there is an image (x x s t ~ X The extracted image is outputted from the image extraction unit 24 as an image to be read.

In the above-mentioned procedure, the images sandwiched between the mark images are sequentially output as images to be read, but when the sub-scanning line YYn is reached, there are no more connected mark images, so the extraction of images to be read ends. .

It should be noted that the determination as to whether or not the mark images are connected is made using the following equation (3) using a known technique, for example, as detailed in Japanese Patent Publication No. 60-55868.

XX<+-r) kb≦XX+*- X xxlkmIX, which indicates the X coordinate of a pair of points of change from color characteristic "0. to color characteristic rj" and change points from color characteristic "j" to color characteristic "0" in the previous scanning line YYi-1;
Xlkk indicates the X coordinate of a pair of points of change from the color characteristic "0" of the scanning line Yi(7) to the same color characteristic "1j" and from the color characteristic "rj" to the color characteristic rOJ, respectively. However, if the full text reading mode is set in the extraction mode switching unit 23, the above process is not performed and the image input unit 1
All the images outputted from the image extraction section 24 are outputted as images to be read.

Note that when the document or book to be read is achromatic, the image extraction section 24 is provided with an achromatic image extraction means, and the images output from the image extraction section 24 are only achromatic images. It is possible to remove the influence of mark images.

The achromatic image extraction means uses a known method, for example, R, G
, B are all at nearly the same level.

Further, a binarization means is connected to the achromatic color extraction means, and
The digitized image is used as the image to be read. Note that the threshold value for binarization may be determined using any known method.

If you specify "Bibliographical matters" as the reading target and input the document shown in Figure 5, the image surrounded by the mark mt will be output as the reading target image, and characters will be cut out and recognized from the image. After that, the voice is synthesized and the reading voice is output.

Similar processing is performed when a "summary or title of chapter 1 section 1" is specified as the reading target.

Note that the position, size, number of colors, and shape of the mark to be added or printed on the color command learning paper, document, or book are not limited to those described above.

Further, a mark is added to the paper to determine whether it is a document, a book, or a color command learning paper, and a function for identifying the presence or absence of the mark is added to the learning/reading mode switching unit 211 of the color command setting unit 21. This allows the mode switching operation to be automated.

Note that the mark is, for example, a black line segment of a predetermined length, a predetermined width, and a predetermined number written in an area near the top side of the paper.

In this embodiment, the learning/reading mode switching section 211 and the color command learning section 212 of the color command setting section 21 are used to learn the marks and characters on the color command paper and set the color characteristics and reading area. In the above explanation, marks added or printed on documents and books were used for the reading area, but color-coded characters of documents and books were used, and the mark identification process of the image extraction unit 24 described above was performed using color-coded characters. Configured to extract images is also good.

Note that the processing after image extraction of the reading area is not limited to that described above and to that shown in FIG.

〔Effect of the invention〕

As explained in detail above, according to the present invention, a color mark is added or printed on a desired part of a document or book,
This reading processing device allows you to freely select the text or the full text of the color marked part of the document or book using voice guidance and voice recognition. Once a document or book has been marked with addendum marks, it can be read in a way similar to the way healthy people read documents and books on a daily basis.
Helps improve the reading environment for people with disabilities such as blind people and physically disabled people. Further, an excellent effect can be obtained in that it is possible to provide a reading processing device with the same environment as an auditory reading environment for a healthy person.

[Brief explanation of drawings]

FIG. 3 is a block diagram showing the configuration of the optical system; FIG. 3 is an explanatory diagram of color characteristics; FIG.
Figure 4(b) is an explanatory diagram of the mark density range, and Figure 4(a)
, cb) and (c) are diagrams each showing an example of a color command learning sheet, FIG. 5 is a diagram showing an example of an input document, and FIG. 6 is a voice command. FIG. 7 is a diagram showing an example of processing using voice guidance, and is an explanatory diagram of image extraction. In the figure, 1... image input section, 2... image processing section,
3...Character recognition processing unit, 4...Language processing unit, 5
...Speech synthesis section 1.6...Speech output section, 21.
...Color command setting section, 211... Learning/reading mode switching section, 212... Color command learning section, 22... Reading mode specifying section, 221... Audio input section, 222 ...Speech recognition unit, 223...Reading mode setting unit, 23...Extraction mode switching unit, 24
...Image extraction section. 1LLs Shinto ritual C Riura summer λ Peng θ Ku ≦ Ruzufu O-I Fu eyes 2nd figure Color eyebrows - Late Moe eyes 3rd figure - 11 → Lord 11'' (XX) Enter-j Jj, Louis 枦 Jf , i min ♂ [[n] Fig. 5 fp comment '7 g door 1
ri f'j row, pull column Fig. 6

Claims (1)

[Scope of Claims] A reading processing device that includes an image input section, an image processing section, a character recognition section, a language processing section, a speech synthesis section, etc., and outputs a reading voice based on a result of character recognition, comprising: the image input section. as written on the paper,
Using an image input unit that outputs information corresponding to one or both of the shading and color of images such as figures, photographs, and/or marks, the image processing unit inputs predetermined color characteristics and reading area names into a pair of reading images. A color command setting section for setting information, a reading mode specifying section for specifying a reading area name through voice interaction processing, and a color characteristic for reading color characteristics of reading information based on the reading area name specified in the reading mode specifying section. and an image extraction section that extracts a document image within a mark area written on a document or book based on the read color characteristics. Reading processing device.