JPH10320109A - Display device, enlarged display support method, and voice support method - Google Patents

Abstract

(57) [Problem] To provide a display device that can communicate a user's intention to perform an enlarged display of a desired image without performing a manual operation. A display device has an enlarged display support function for enlarging and displaying an image displayed on a CRT 6 according to a user's intention. In an enlarged display process using the enlarged display support function, display support is provided. The detection result of the line-of-sight detection unit of the device 4 is captured via the interface 5, and based on the detection result of the captured line-of-sight detection unit, an area on the screen of the CRT 6 where the user's line of sight is directed for a certain time or more is specified. The specified area is enlarged at a predetermined magnification, image data indicating the enlarged area is transferred to the LCD 52 of the display support device 4 via the interface 5, and the image of the enlarged area is displayed on the LCD 52.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device for assisting communication of a user's intention, an enlarged display support method for enlarging and displaying an image displayed on display means according to the user's intention, The present invention also relates to a voice support method for voice a character string displayed on a display means according to a user's will.

[0002]

2. Description of the Related Art Generally, a display device having a function of assisting communication of a user's intention includes a display such as a cathode ray tube (CRT) and a liquid crystal display, and a mouse and a keyboard for indicating a position on the screen of the display. Some are combined with manually operated input devices such as For example, in this display device, when a part of an image such as a character or a symbol displayed on a display screen is enlarged and displayed by an enlarged display function, a display position (area) of an image that the user desires to enlarge is displayed. ) And the enlargement magnification are designated by operating the input device. An image to be enlarged and displayed and an enlargement magnification of the image are designated based on an instruction from the operation of the input device, and the designated image is enlarged and displayed on the display screen at the designated enlargement magnification.
As described above, the intention of the user is transmitted by the user manually operating the input device.

Further, as a device for assisting communication of a user's intention, a device using a line-of-sight detection device for detecting a line of sight of a user has appeared. For example, this device is incorporated in a finder of a camera. In a device incorporated in this camera, an area in which the user's line of sight is directed is detected by a line-of-sight detection device in a display area of the object in the viewfinder, and an object displayed in the area where the line of sight is directed is detected. It is configured to be focused. As described above, it is possible to focus on a subject desired by the user by detecting a region where the user's gaze is directed by the gaze detection device.

[0004]

However, in the above-described conventional display device, the user's intention is transmitted by manually operating an input device such as a mouse and a keyboard, so that the user can manually operate the input device. When a manual operation cannot be performed, such as a state in which the user cannot use, the user's intention regarding the enlarged display function cannot be transmitted.

A device that supports the transmission of a user's intention using a line-of-sight detection device can transmit the user's intention without performing manual operation. Is applied only to assist the transmission of the user's intention with respect to the control items related to focusing, and other items, such as the above-described display device, using the line-of-sight detection device to determine the user's will No device has been developed for communication.

A first object of the present invention is to provide a display device and an enlarged display support method capable of communicating a user's intention to perform an enlarged display of a desired image without performing a manual operation. Is to provide.

A second object of the present invention is to provide a display device and a voice support method which can communicate a user's intention so as to voice a character desired by the user without manual operation. To provide.

[0008]

According to the first aspect of the present invention,
In a display device having a support function for supporting communication of a user's intention, a line of sight detecting means for detecting the line of sight of the user, and the line of sight of the user is directed based on a detection result of the line of sight detection means Area specifying means for specifying an area on the screen of the display means, enlargement processing means for enlarging the specified area at a predetermined magnification, and arranged at a position visible to the user to display the enlarged area And an enlarged image display means.

According to a second aspect of the present invention, in the display device according to the first aspect, the area specifying means determines an area in which the user's gaze is directed for a predetermined time or more based on a detection result of the gaze detection means. It is characterized by specifying.

According to a third aspect of the present invention, in the display device according to the first aspect, the line-of-sight detection means and the enlarged image display means are integrally configured as an enlarged display support means.

According to a fourth aspect of the present invention, in the display device according to the first aspect, a state in which the enlarged display processing of the specified area is temporarily stopped ignoring the detection result of the line-of-sight detection means can be released. It is characterized by comprising state setting means for setting.

According to a fifth aspect of the present invention, there is provided a display device having a support function for supporting communication of a user's intention,
Gaze detection means for detecting the gaze of the user, area identification means for identifying an area on a screen of a display means to which the gaze of the user is directed based on the detection result of the gaze detection means, Recognizing means for recognizing an image displayed in the selected area, an enlarging processing means for selecting an image in the specified area according to the recognition result of the recognizing means, and enlarging the selected image, An enlarged image display means is provided at a position visible to the user and displays the enlarged image.

According to a sixth aspect of the present invention, in the display device according to the fifth aspect, the area specifying means determines an area in which the user's gaze is directed for a predetermined time or more based on a detection result of the gaze detection means. It is characterized by specifying.

According to a seventh aspect of the present invention, in the display device according to the fifth aspect, the line-of-sight detection means and the enlarged image display means are integrally formed as an enlarged display support means.

According to an eighth aspect of the present invention, in the display device of the fifth aspect, the state in which the enlarged display processing of the image of the specified area is temporarily stopped ignoring the detection result of the line of sight detection means is released. It is characterized by comprising state setting means for setting possible.

According to a ninth aspect of the present invention, there is provided a display device having a support function for supporting communication of a user's intention,
Gaze detection means for detecting the gaze of the user, area identification means for identifying an area on a screen of a display means to which the gaze of the user is directed based on the detection result of the gaze detection means, And voice conversion processing means for converting a character string displayed in the displayed area into voice.

According to a tenth aspect of the present invention, in the display device according to the ninth aspect, the voice conversion operation by the voice conversion processing unit is temporarily stopped by ignoring the detection result of the visual line detection unit. An independent state setting unit is provided for setting a state in which a detection result of the line-of-sight detection unit and an audio conversion operation by the audio conversion processing unit are independent from each other so as to be cancelable.

According to an eleventh aspect of the present invention, in the display device according to the tenth aspect, the area specifying means determines an area in which the user's gaze is directed for a predetermined time or more based on a detection result of the gaze detection means. It is characterized by specifying.

According to a twelfth aspect of the present invention, there is provided an enlarged display support method for enlarging and displaying an image displayed on a display means in accordance with a user's will, wherein the step of detecting a line of sight of the user is provided. A step of specifying an area on the screen of the display means to which the user's line of sight is directed based on the result of detection of the line of sight; a step of enlarging the specified area by a predetermined magnification; Displaying the enlarged area on enlarged image display means arranged at a position.

According to a thirteenth aspect of the present invention, in the enlarged display support method according to the twelfth aspect, it is possible to cancel the state in which the enlarged display processing of the specified area is temporarily stopped by ignoring the line-of-sight detection result. It is characterized by including the step of setting.

According to a fourteenth aspect of the present invention, there is provided an enlarged display support method for enlarging and displaying an image displayed on a display means according to a user's intention, wherein the step of detecting a line of sight of the user is provided. A step of specifying an area on a screen of a display unit to which the user's line of sight is directed based on a result of the line of sight detection; a step of recognizing an image displayed in the specified area; Selecting an image in the specified area according to the above, and enlarging the selected image, and displaying the enlarged image on enlarged image display means arranged at a position visible to the user And characterized in that:

According to a fifteenth aspect of the present invention, in the enlarged display support method according to the fourteenth aspect, the state in which the enlarged display processing of the image of the specified area is temporarily stopped ignoring the detection result of the line of sight is canceled. It is characterized by including the step of setting as possible.

According to a sixteenth aspect of the present invention, there is provided a voice assisting method for selecting and characterizing a character string displayed on a display means according to a user's will, wherein a step of detecting a line of sight of the user is provided. Identifying a region on the screen of the display means to which the user's line of sight is directed based on the result of detection of the line of sight, and converting a character string displayed in the specified region to voice. It is characterized by including.

According to a seventeenth aspect of the present invention, in the voice assisting method according to the sixteenth aspect, the conversion operation from the character string to the voice is temporarily stopped while ignoring the result of detection of the line of sight. Setting a state in which the detection result of the line of sight and the conversion operation to the voice are independent from each other so as to be releasable.

[0025]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram showing the configuration of a first embodiment of the display device of the present invention. Note that in this embodiment, an example of a display device including a personal computer will be described.

As shown in FIG. 1, the display device having the function of supporting the communication of the user's intention is constituted by using a personal computer. The personal computer has a system control program 11 stored in a program memory 10. And a CPU 1 that executes an application program corresponding to the OS on the OS by using the RAM (random access memory) 12 as a work area to execute a corresponding process.
The program memory 10 includes a system control program 1
1 is stored together with the display support device driver 12, and the display support device driver 12
F) Communication control for transmitting / receiving data to / from the display support apparatus 4 via 5 and a program for executing an enlarged display process. The above-described application program and data necessary for executing the application program are sent from the hard disk device (HD) 2 to the hard disk controller (HD).
C) read out via 3. The details of the contents of the display support device driver 12 will be described later.

The CPU 1 has the program memory 1 described above.
0, a RAM 13, a hard disk controller 3, and an interface 5, a display controller 7, a keyboard 8, and a mouse 9 are connected via a system bus 16. The hard disk controller 3 controls writing and reading of data to and from the hard disk 2, and the interface 5 controls an interface with the display support device 4. The display controller 7 performs display control for displaying an execution result of the application by the CPU 1 on the CRT 6. Key signals input from the keyboard 8 or the mouse 9 are transmitted to the CPU 1 via the system bus 16.

Next, the configuration of the display support apparatus 4 will be described with reference to FIG.
This will be described with reference to FIG. FIG. 2 is a diagram schematically showing a main configuration of a display support device provided in the display device of FIG. 1, and FIG. 3 shows a state of a reflection surface in an eyepiece prism provided in the display support device of FIG. FIGS. 4A and 4B are diagrams for explaining the relationship between the gaze detection by the display support device of FIG. 2 and the movement of the user's eyeball.

As shown in FIG. 2, the display support device 4 has a head mount type structure used by being worn on the user's head, and detects the movement of the user's eyeball 56 to detect the user's line of sight. And a video observation unit for guiding the image displayed on the liquid crystal display (LCD) 52 to the user's eyeball 56 so that the user can observe the image.

Specifically, the line-of-sight detection means has an IRED 53 for irradiating the user's eyeball 56 with infrared light in the form of a spot. The light enters the optical lens group 54 via the aperture stop A. The optical lens group 54 converts the incident infrared reflected light into a CCD
The CCD 55 converts the virtual image (Purkinje image) formed by the corneal reflection formed on the imaging surface into an electric signal and outputs the electric signal. As shown in FIG. 4, this virtual image is an image 72 of the pupil 71 of the infrared light emitted from the IRED 53 due to corneal reflection, and the center of the pupil 71 and the virtual image 72
Changes in proportion to the rotation angle of the eyeball. Therefore, image processing is performed using an electric signal indicating a virtual image from the CCD 55, and the direction of the user's line of sight is detected based on the result of the image processing.

The line-of-sight observation means includes, specifically, a liquid crystal display device 52 for displaying an image transferred via the interface 5 and two reflection surfaces 51a, 5a, as will be described later.
An image displayed on the liquid crystal display device 52 is guided to the user's eyeball 56 via the eyepiece 51. Specifically, the optical image incident on the inside of the eyepiece 51 from the liquid crystal display device 52 is reflected by the reflection surface 51a and then again reflected by the reflection surface 51b and guided to the eyeball 56,
The user observes the image projected on the display area M of the reflection surface 51b, as shown in FIG.

Next, the configuration of the display support device driver 12 stored in the program memory 10 will be described.

The display support device driver 12 initializes a detection condition of the line-of-sight detection means of the display support device 4 and initializes the interface 5 and a detection result of the line-of-sight detection device of the display support device 4. Interface 5
A module for specifying an area on the screen of the CRT 6 to which the user's line of sight is directed for a certain time or more based on the detection result of the captured line of sight detection means, and a module for capturing the specified area. A module for enlarging at a predetermined magnification and transferring image data indicating the enlarged area to the LCD 52 of the display support device 4 via the interface 5.

Next, an enlarged display process using the display support device 4 will be described with reference to FIG. FIG. 5 is a flowchart showing a procedure of an enlarged display process using the display support device in the display device of FIG.

When performing an enlarged display process using the display support device 4, first, a mode for executing the enlarged display process using the display support device 4 is set, and the display support device 4 is placed on the head of the user. Be attached.

When the mode for executing the enlarged display process using the display support device 4 is set, the CPU 1 executes the process in accordance with the display support device driver 12. First, FIG.
As shown in (5), in step S21, the viewpoint conditions are initialized, and in step S22, the interface 5 is initialized. In the initialization of the viewpoint conditions, the operation environment of the display support device 4 is set. To set up this operating environment,
A process of writing an initial value of the viewpoint data to the viewpoint data area 15 of the RAM 13 is included.
This is a value for ensuring consistency between the reference position on the screen of the RT 6 and the detection result of the line-of-sight detection unit of the display support device 4. In the initialization of the interface 5, an operation environment for transmitting and receiving data to and from the display support device 4 with respect to the interface 5 is set.

Then, the process proceeds to a step S23, wherein the RAM 1
The viewpoint data held in the third viewpoint data area 15 is read, and in the following step S24, the user's line of sight is directed to any position in the screen of the CRT 6 based on the viewpoint data read from the viewpoint data area 15 of the RAM 13. Is calculated, and an area on the screen of the CRT 6 where the user's line of sight is directed is specified based on the calculated position. The specified area is represented by position data, and the position data is temporarily stored in the position data area 14 of the RAM 13. At the initial stage, an area on the screen of the CRT 6 to which the user's line of sight is directed is specified based on the initial value of the above-mentioned viewpoint data. It is possible to prevent the viewpoint data from fluctuating greatly due to (line-of-sight variation) and not being uniquely determined.

Then, in step S25, an enlargement process for enlarging the area indicated by the position data held in the position data area 14 of the RAM 13 at a predetermined magnification and generating image data indicating the enlarged area is performed. The generated image data is temporarily stored in the RAM 13. In a succeeding step S26, the image data held in the RAM 13 is transferred to the LCD 52 of the display support device 4 via the interface 5.
The LCD 52 displays an image indicated by the transferred image data, and the displayed image is guided to the user's eyeball 56 via the eyepiece 51. Therefore, the user observes the image displayed on the LCD 5, that is, the image in which the area where the user's line of sight is directed is enlarged.

Then, the process proceeds to a step S27 to determine whether or not the user has instructed the end of the enlarged display processing.
When the end of the enlarged display processing is not instructed, the process proceeds to step S29, in which the detection result from the line-of-sight detection unit of the display support device 4 is captured via the interface 5, and based on the detection result, the user's line of sight is kept for a certain time or more. Calculate the point of view point that is pointed. The calculated viewpoint position is held in the viewpoint data area 15 of the RAM 13 as viewpoint data. Then, the process returns to step S23 again to execute the processing from step S23.

On the other hand, when the end of the enlarged display processing is instructed, the process proceeds to step S28, the end processing is performed, and this processing ends.

As described above, the region on the screen of the CRT 6 to which the user's line of sight is directed for a certain time or more is specified based on the detection result of the line of sight detection means of the display support device 4, and the specified region is enlarged at a predetermined magnification. Then, since the enlarged area is displayed on the LCD 52 of the display support device 4, it is possible to communicate the user's intention to perform the enlarged display of the image desired by the user without performing a manual operation. .

(Second Embodiment) Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart illustrating a procedure of an enlarged display process using the display support device according to the second embodiment of the display device of the present invention.

The present embodiment has the same configuration as the above-described first embodiment, but differs from the first embodiment in that an image displayed in an area specified by a gaze detection result is recognized, and An enlargement process for selecting an image portion to be enlarged from the specified area according to the recognition result and enlarging the selected image portion at a predetermined enlargement ratio, and a function of setting the image enlargement ratio to an arbitrary magnification are added. Is different.

When the mode for executing the enlarged display processing using the display support device 4 is set, first, as shown in FIG. 6, an initial condition is input in step S31. In inputting the initial conditions, setting such as the magnification of the designated image is performed. Next, the process proceeds to step S32, in which the viewpoint conditions are initialized, and in step S33, the interface 5 is initialized.

Then, the process proceeds to a step S34, wherein the RAM 1
The viewpoint data held in the third viewpoint data area 15 is read, and in the following step S35, the user's line of sight is directed to any position in the screen of the CRT 6 based on the viewpoint data read from the viewpoint data area 15 of the RAM 13. Is calculated, and an area on the screen of the CRT 6 where the user's line of sight is directed is specified based on the calculated position. The specified area is represented by position data, and the position data is temporarily stored in the position data area 14 of the RAM 13. At the initial stage, an area on the screen of the CRT 6 to which the user's gaze is directed is specified based on the initial value of the viewpoint data described above. It is possible to prevent a situation in which the value fluctuates greatly and is not uniquely determined.

Then, in step S36, it is determined whether most of the images in the area indicated by the position data held in the position data area 14 of the RAM 13 are text images. Specifically, this determination is made according to whether or not an image of a predetermined ratio (for example, 80%) or more of the images in the area indicated by the position data is a text image.
If most of the image is a text image, the process proceeds to step S37 to perform an enlargement process of enlarging the text image in the area indicated by the position data at the set magnification and generating image data indicating the enlarged image. The generated image data is temporarily stored in the RAM 13.

If most of the image is not a text image, the flow advances to step S38 to determine whether most of the image in the area indicated by the position data held in the position data area 14 of the RAM 13 is a bitmap image.
This determination is performed in the same manner as in step S36. When most of the image is a bitmap image, the process proceeds to step S39, in which an enlargement process of enlarging the bitmap image in the area indicated by the position data at the set magnification and generating image data indicating the enlarged image is performed. The generated image data is temporarily stored in the RAM 13.

If the majority of the image is not a bitmap image, the flow advances to step S40 to determine whether the majority of the image in the area indicated by the position data held in the position data area 14 of the RAM 13 is a draw image. This determination is performed in the same manner as in step S36. If most of the images are draw images, step S41
The enlargement process is performed to enlarge the draw image in the area indicated by the position data at the set magnification and generate image data indicating the enlarged image. The generated image data is temporarily stored in the RAM 13.

When most of the images are not draw images, a text image, a bitmap image and a draw image are mixed in the specified area, or the ratio of the text image, the bitmap image and the draw image is different. When it is determined that the ratio is less than the predetermined ratio, the process proceeds to step S42, in which an area indicated by the position data is enlarged by the set magnification, and enlargement processing for generating image data of the enlarged area is performed.
The generated image data is temporarily stored in the RAM 13.

After the enlarging process in any of steps S37, 39, 41, and 42, the process proceeds to step S43, where RA
The image data held in M13 is displayed on the LC of the display support device 4.
The data is transferred to D52 via the interface 5. LCD5
Reference numeral 2 denotes an image indicated by the transferred image data, and the displayed image, that is, C to which the user's eyes are directed
The image obtained by enlarging the image of the specified area on the screen of the RT 6 is guided to the user's eyeball 56 via the eyepiece 51.
Therefore, the user observes an enlarged image of the image at the position on the CRT 6 to which the user looks.

Then, the process proceeds to a step S44, and it is determined whether or not the user has instructed the end of the enlarged display processing.
When the end of the enlarged display processing is not instructed, the process proceeds to step S46, in which the detection result from the line-of-sight detection unit of the display support device 4 is captured via the interface 5, and based on the detection result, the user's line of sight is kept for a certain time or more. Calculate the point of view point that is pointed. The calculated viewpoint position is held in the viewpoint data area 15 of the RAM 13 as viewpoint data. Then, the process returns to step S34 again to execute the processing from step S34.

On the other hand, when the end of the enlarged display processing is instructed, the process proceeds to step S45, where the end processing is performed, and the present processing ends.

As described above, the area on the screen of the CRT 6 to which the user's line of sight has been directed for a certain period of time or more is specified based on the detection result of the line of sight detection means of the display support device 4, and is displayed in the specified area. Since the image is enlarged at a predetermined magnification and the enlarged image is displayed on the LCD 52 of the display support device 4, the user's intention is to perform an enlarged display of the image desired by the user without performing manual operation. Communication can be achieved. Also, the image displayed in the area specified by the gaze detection result is recognized, an image part to be enlarged is selected from the area specified in accordance with the recognition result, and the selected image part is enlarged at a predetermined magnification. Since the image is enlarged, an enlarged image excluding an extra image can be observed.

(Third Embodiment) Next, a third embodiment of the present invention will be described with reference to FIG. FIG. 7 is a flowchart illustrating a procedure of an enlarged display process using the display support device according to the third embodiment of the display device of the present invention.

The present embodiment has the same configuration as the above-described first embodiment, but is different from the first embodiment in that the enlarged display processing of the area specified by ignoring the line-of-sight detection result is temporarily performed. The difference is that the stop state is set to be releasable and the magnification is set to an arbitrary magnification.

When the mode for executing the enlarged display processing using the display support device 4 is set, first, as shown in FIG. 7, an initial condition is input in step S51. In inputting the initial conditions, setting such as the magnification of the designated image is performed. Next, the process proceeds to step S52, where the viewpoint conditions are initialized, and in the following step S53, the interface 5 is initialized.

Then, the process proceeds to a step S54, wherein the RAM 1
The viewpoint data held in the third viewpoint data area 15 is read, and in the following step S55, the user's line of sight is directed to any position in the screen of the CRT 6 based on the viewpoint data read from the viewpoint data area 15 of the RAM 13. Is calculated, and an area on the screen of the CRT 6 where the user's line of sight is directed is specified based on the calculated position. The specified area is represented by position data, and the position data is temporarily stored in the position data area 14 of the RAM 13. At the initial stage, an area on the screen of the CRT 6 to which the user's gaze is directed is specified based on the initial value of the viewpoint data described above. It is possible to prevent a situation in which the value fluctuates greatly and is not uniquely determined.

Then, in step S56, the area indicated by the position data held in the position data area 14 of the RAM 13 is enlarged at a predetermined magnification, and enlargement processing for generating image data indicating the enlarged area is performed. The generated image data is temporarily stored in the RAM 13. In a succeeding step S57, the image data held in the RAM 13 is transferred to the LCD 52 of the display support device 4 via the interface 5.
The LCD 52 displays an image indicated by the transferred image data, and the displayed image is guided to the user's eyeball 56 via the eyepiece 51. Therefore, the user can use the LCD 52
, That is, an image obtained by enlarging an area where the user's gaze is directed.

Then, the process proceeds to a step S58 to determine whether or not the user has instructed the end of the enlarged display processing.
When the end of the enlargement display processing is not instructed, the process proceeds to step S60, and it is determined whether or not the enlargement processing is performed according to the movement of the viewpoint. If the enlargement process is to be performed in accordance with the movement of the viewpoint, the flow returns to step S58 again, and steps S58 and S60 are performed until the end of the enlargement display process is instructed or until it is determined that the enlargement process is to be performed in accordance with the movement of the viewpoint. A state is set in which the enlarged display processing of the specified area is temporarily stopped by ignoring the line-of-sight detection result by looping.

If it is determined that the enlargement process is to be performed in accordance with the movement of the viewpoint, the process proceeds to step S61, where the detection result from the line-of-sight detection means of the display support device 4 is fetched via the interface 5 and used based on the detection result. The viewpoint position at which the user's gaze is directed for a certain period of time or more is calculated. The calculated viewpoint position is held in the viewpoint data area 15 of the RAM 13 as viewpoint data. And, again, step S54
Then, the process of enlarging the area specified according to the movement of the viewpoint is executed.

On the other hand, when the end of the enlarged display processing is instructed in step S58, the process proceeds to step S59, where the end processing is performed, and the present processing ends.

As described above, according to the determination result of step S60, a state is set in which the enlarged display processing of the specified area is temporarily stopped while ignoring the detection result of the line of sight. You can rest and reduce eye fatigue.

It is possible to apply the state in which the enlarged display processing of the specified area is temporarily stopped while ignoring the detection result of the line of sight to the second embodiment described above. In this case, By inserting a step (step S60) of determining whether or not to perform the enlargement process in accordance with the movement of the viewpoint after step S44 of FIG. 6 in the second embodiment, Thus, the user can temporarily rest his / her eyes, thereby reducing eyestrain.

(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described with reference to FIGS. FIG. 8 is a block diagram showing the configuration of the fourth embodiment of the display device of the present invention, and FIG. 9 is a flowchart showing the procedure of the voice processing using the voice support device in the display device of FIG. In this embodiment, the first embodiment described above is used.
As in the embodiment, it is configured using a personal computer, and the same components are denoted by the same reference numerals, and description thereof will be simplified or omitted.

The display device according to the present embodiment is a display device having a voice support function for selecting and characterizing a character string displayed on the display means according to the user's intention, and is shown in FIG. Program memory 10, RA
M13, a hard disk controller 3, an interface (I / F) 22, a display controller 7, an audio controller 25, a keyboard 8, a mouse 9, and a CPU 1 connected via a system bus 16.

The program memory 10 stores a system control program 11 and a voice support device driver 23. The voice support device driver 23 exchanges data with the voice support device 21 via the interface 22. It consists of a program for performing communication control for transmitting and receiving and voice processing. The details of the content of the driver 23 for the voice support device will be described later.

The configuration of the speech support device 21 is the same as that of the enlarged display support device 4 of the first embodiment (shown in FIG. 2), and the description of the configuration is omitted.

The voice support device driver 23 stored in the program memory 10 includes a module for initializing the detection conditions of the line-of-sight detecting means of the voice support device 21 and the interface 22, and a display support device. A module for capturing the detection result of the line-of-sight detection unit of the device 4 via the interface 22, and an area on the screen of the CRT 6 where the user's line of sight is directed for a certain time or more based on the detection result of the captured line-of-sight detection unit And a module for performing a voice conversion process of converting a character string displayed in the specified area into voice data.

When the mode for executing the speech conversion process using the speech support device 21 is set, first, as shown in FIG. 9, the viewpoint conditions are initialized in step S71, and the interface 22 is initialized in step S72. Is initialized.

Then, the process proceeds to a step S73, wherein the RAM 1
The viewpoint data held in the third viewpoint data area 15 is read, and in the following step S74, the user's line of sight is directed to any position in the screen of the CRT 6 based on the viewpoint data read from the viewpoint data area 15 of the RAM 13. Is calculated, and an area on the screen of the CRT 6 where the user's line of sight is directed is specified based on the calculated position. The specified area is represented by position data, and the position data is temporarily stored in the position data area 14 of the RAM 13. At the initial stage, an area on the screen of the CRT 6 to which the user's gaze is directed is specified based on the initial value of the viewpoint data described above. It is possible to prevent a situation in which the value fluctuates greatly and is not uniquely determined.

Then, in step S75, a character string displayed in an area indicated by the position data held in the position data area 14 of the RAM 13 is extracted, and a voice conversion process for converting the extracted character string into voice data is performed. This audio data is temporarily stored in the RAM 13. In the following step S76,
The audio data held in the RAM 13 is transferred to the audio controller 25, and the audio data transferred by the audio controller 25 is converted into an analog audio signal. This analog audio signal is output to the speaker 24, and the speaker 24 emits sound. That is, a voice representing a character string displayed in the area of the CRT 6 where the user's line of sight is directed is emitted. Therefore, the user hears the sound emitted from the speaker 24, that is, the sound representing the character string displayed in the area where the user's gaze is directed.

Next, the process proceeds to a step S77, and it is determined whether or not the user has instructed the end of the voice processing. If the end of the voice processing is not instructed, the process proceeds to step S79, in which the detection result from the visual line detection means of the voice support device 21 is captured via the interface 22, and the user's visual line is kept for a predetermined time based on the detection result. The position of the viewpoint pointed at above is calculated. The calculated viewpoint position is held in the viewpoint data area 15 of the RAM 13 as viewpoint data. Then, the process returns to step S73 again to execute the processing from step S73.

On the other hand, when the end of the voice processing is instructed, the process proceeds to step S78, where the end processing is performed, and the present processing is ended.

As described above, the area on the screen of the CRT 6 to which the user's line of sight has been directed for a certain period of time or more is specified based on the detection result of the line-of-sight detection means of the speech support device 21 and displayed in the specified area. Since the character string is vocalized, the intention of the user can be communicated so that the user can vocalize the desired character without manual operation.

(Fifth Embodiment) Next, a fifth embodiment of the present invention will be described with reference to FIG. FIG.
0 is a flowchart showing the procedure of the voice processing using the voice support apparatus according to the fifth embodiment of the display device of the present invention.

The present embodiment has the same configuration as the above-described fourth embodiment, but differs from the fourth embodiment in that the voice conversion operation is temporarily stopped while ignoring the line-of-sight detection result. Thus, the difference is that the state in which the gaze detection result and the operation of converting to voice are independent can be set to be releasable, and the condition such as the volume is set to an arbitrary value.

More specifically, when the mode for executing the voice conversion process using the voice support device 21 is set, first,
As shown in FIG. 10, an initial condition is input in step S81. In the input of the initial condition, the speaker 24
Is input as needed. Next, in step S82, the viewpoint conditions are initialized, and then in step S8.
In step 3, the interface 22 is initialized.

Then, the process proceeds to a step S84, wherein the RAM 1
The viewpoint data held in the third viewpoint data area 15 is read, and in the following step S85, the user's line of sight is directed to any position in the screen of the CRT 6 based on the viewpoint data read from the viewpoint data area 15 of the RAM 13. Is calculated, and an area on the screen of the CRT 6 where the user's line of sight is directed is specified based on the calculated position. The specified area is represented by position data, and the position data is temporarily stored in the position data area 14 of the RAM 13. At the initial stage, an area on the screen of the CRT 6 to which the user's gaze is directed is specified based on the initial value of the viewpoint data described above. It is possible to prevent a situation in which the value fluctuates greatly and is not uniquely determined.

Then, in step S86, a character string displayed in the area indicated by the position data held in the position data area 14 of the RAM 13 is extracted, and voice processing for converting the extracted character string into voice data is performed. This audio data is temporarily stored in the RAM 13. In a succeeding step S87,
The audio data held in the RAM 13 is transferred to the audio controller 25, and the audio data transferred by the audio controller 25 is converted into an analog audio signal. This analog audio signal is output to the speaker 24, and the speaker 24 emits sound. That is, a voice representing a character string displayed in the area of the CRT 6 where the user's line of sight is directed is emitted. Therefore, the user hears the sound emitted from the speaker 24, that is, the sound representing the character string displayed in the area where the user's gaze is directed.

Next, the process proceeds to a step S88, and it is determined whether or not the user has instructed the end of the voice processing. When the end of the voice processing is not instructed, the process proceeds to step S90, and it is determined whether or not the voice processing is performed in accordance with the movement of the viewpoint. If the voice processing is to be performed in accordance with the movement of the viewpoint, the process returns to step S88 again, and until the termination of the voice processing is instructed or until it is determined that the voice processing is to be performed in accordance with the movement of the viewpoint, steps S88 and S88 are performed. By looping step S90, a state in which the voice processing is temporarily stopped while ignoring the line-of-sight detection result is maintained.

If it is determined that the voice processing is to be performed in accordance with the movement of the viewpoint, the flow advances to step S91, and the detection result from the line-of-sight detecting means of the voice
2, and calculates a viewpoint position at which the user's line of sight is directed for a certain time or more based on the detection result. The calculated viewpoint position is held in the viewpoint data area 15 of the RAM 13 as viewpoint data. Then, the process returns to step S84 again, and performs a voice conversion process of converting the character string in the area specified according to the movement of the viewpoint into voice data.

On the other hand, when the end of the voice processing is instructed in step S88, the process proceeds to step S89,
End processing is performed, and this processing ends.

As described above, the state in which the voice processing is temporarily stopped ignoring the detection result of the line of sight according to the determination result of step S88 is maintained, so that the user can temporarily rest his eyes. Can reduce eye fatigue.

[0085]

As described above, according to the display device of the first aspect, the line of sight detecting means for detecting the line of sight of the user and the line of sight of the user based on the detection result of the line of sight detecting means. Area specifying means for specifying an area on the screen of the display means being displayed, enlargement processing means for enlarging the specified area at a predetermined magnification, and enlargement arranged at a position visible to the user and displaying the enlarged area Image display means,
It is possible to communicate the user's intention to perform an enlarged display of the image desired by the user without performing a manual operation.

According to the display device of the second aspect, the area specifying means specifies the area in which the user's gaze is directed for a predetermined time or more based on the detection result of the gaze detecting means.
A region where the user's gaze is directed can be specified with high accuracy.

According to the display device of the third aspect, the line-of-sight detection means and the enlarged image display means can be integrally configured as enlarged display support means.

According to the display device of the fourth aspect, since the state setting means for setting the state in which the enlarged display processing of the specified area is temporarily stopped ignoring the detection result of the line-of-sight detection means is set to be releasable. In addition, the user can temporarily rest their eyes, thereby reducing eyestrain.

According to the display device of the fifth aspect, the line of sight detecting means for detecting the line of sight of the user and the area on the screen of the display means to which the line of sight of the user is directed based on the detection result of the line of sight detecting means An area specifying means for specifying, a recognition means for recognizing an image displayed in the specified area, and an image in the specified area according to the recognition result of the recognition means are selected, and the selected image is selected. Since there is provided an enlargement processing unit for enlarging and an enlarged image display unit for displaying an enlarged image arranged at a position visible to the user, it is possible to enlarge and display an image desired by the user without performing a manual operation. In this way, it is possible to convey the user's intention as well as to observe an enlarged image excluding an extra image.

According to the display device of the sixth aspect, since the area specifying means specifies an area in which the user's line of sight is directed for a predetermined time or more based on the detection result of the line of sight detecting means,
A region where the user's gaze is directed can be specified with high accuracy.

According to the display device of the seventh aspect, the line-of-sight detection means and the enlarged image display means can be integrally configured as enlarged display support means.

[0092] According to the display device of the eighth aspect, the state setting means for ignoring the detection result of the line-of-sight detection means and setting the state in which the enlarged display processing of the image of the specified area is temporarily stopped to be releasable. , The user can temporarily rest their eyes, and can reduce eye fatigue.

According to the display device of the ninth aspect, the line-of-sight detecting means for detecting the line of sight of the user, and the area on the screen of the display means to which the line of sight of the user is directed based on the detection result of the line-of-sight detecting means Is provided, and a voice conversion processing means for converting a character string displayed in the specified area into a voice is provided, so that a character desired by the user is converted into a voice without manual operation. In this way, the intention of the user can be communicated.

According to the display device of the tenth aspect, the detection result of the line-of-sight detection means and the sound are set so that the sound conversion operation by the sound conversion processing means is temporarily stopped while ignoring the detection result of the line-of-sight detection means. Since there is provided the independent state setting means for setting the state independent of the voice conversion operation by the conversion processing means so as to be releasable, the user can temporarily rest his eyes and reduce eyestrain.

According to the display device of the eleventh aspect, the area specifying means specifies an area in which the user's line of sight has been directed for a predetermined time or more based on the detection result of the line of sight detecting means. It is possible to specify the area to be directed with high accuracy.

According to the enlarged display support method of the twelfth aspect, the step of detecting the user's line of sight and the area on the screen of the display means to which the user's line of sight is directed based on the result of the line of sight detection Performing a manual operation, including a step of performing the operation, a step of enlarging the specified area at a predetermined magnification, and a step of displaying the enlarged area on an enlarged image display unit arranged at a position visible to the user. Instead, it is possible to communicate the user's intention so that the image desired by the user is enlarged and displayed.

According to the enlarged display support method of the thirteenth aspect, the method includes the step of setting the state in which the enlarged display processing of the specified area is temporarily stopped ignoring the detection result of the line of sight so as to be releasable. The person can rest his eyes temporarily, which can reduce eye fatigue.

According to the fourteenth aspect of the present invention, the step of detecting the user's line of sight and the area on the screen of the display means to which the user's line of sight is directed based on the result of the line of sight detection. Performing a step of recognizing an image displayed in the specified area, selecting an image in the specified area according to the recognition result, and enlarging the selected image. Displaying an enlarged image on an enlarged image display means arranged at a visible position,
It is possible to convey the user's intention to perform an enlarged display of an image desired by the user without performing a manual operation, and it is possible to observe an enlarged image excluding an extra image.

According to the enlarged display support method of the fifteenth aspect, the step of setting the state of temporarily stopping the enlarged display processing of the image of the specified area ignoring the detection result of the line of sight is set to be releasable. In addition, the user can temporarily rest their eyes, thereby reducing eyestrain.

According to the speech support method of the present invention, the step of detecting the line of sight of the user and the area on the screen of the display means to which the line of sight of the user is directed are specified based on the result of the detection of the line of sight. And a step of converting a character string displayed in the specified area into a voice, so that the user's will can be vocalized to the desired character without manual operation. Communication can be achieved.

According to the speech support method of the seventeenth aspect, the detection result of the line of sight and the speech are converted so that the conversion operation from the character string to the speech is temporarily stopped while ignoring the detection result of the line of sight. A step of setting the state independent of the conversion operation to be releasable, so that the user can temporarily rest their eyes and reduce eye fatigue.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of a display device of the present invention.

FIG. 2 is a diagram schematically showing a configuration of a main part of a display support device provided in the display device of FIG. 1;

FIG. 3 is a diagram showing a state of a reflecting surface in an eyepiece prism provided in the display support device of FIG. 2;

FIG. 4 is a diagram for explaining a relationship between eye gaze detection and movement of a user's eyeball by the display support device of FIG. 2;

FIG. 5 is a flowchart illustrating a procedure of an enlarged display process using a display support device in the display device of FIG. 1;

FIG. 6 is a flowchart illustrating a procedure of an enlarged display process using a display support device according to a second embodiment of the display device of the present invention.

FIG. 7 is a flowchart illustrating a procedure of an enlarged display process using a display support device according to a third embodiment of the display device of the present invention.

FIG. 8 is a block diagram showing a configuration of a fourth embodiment of the display device of the present invention.

FIG. 9 is a flowchart showing a procedure of a speech process using a speech support device in the display device of FIG. 8;

FIG. 10 is a flowchart showing a procedure of a speech process using a speech support device according to a fifth embodiment of the display device of the present invention.

[Explanation of symbols]

 Reference Signs List 1 CPU 4 display support device 5, 22 interface 6 CRT 7 display controller 10 program memory 12 display support device driver 13 RAM 21 voice support device 23 voice support device driver 24 speaker 25 voice controller

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FI G09G 5/00 550 G09G 5/00 550C

Claims (17)

[Claims] 1. A display device having a support function for supporting communication of a user's intention, a gaze detection unit for detecting the gaze of the user, and a gaze of the user based on a detection result of the gaze detection unit. Area specifying means for specifying an area on the screen of the display means to which the user is directed; enlargement processing means for enlarging the specified area at a predetermined magnification; and A display device comprising: an enlarged image display means for displaying a selected area. 2. The apparatus according to claim 1, wherein the area specifying unit specifies an area in which the user's line of sight is directed for a predetermined time or more based on a detection result of the line of sight detecting unit.
The display device according to the above. 3. The display device according to claim 1, wherein said line-of-sight detecting means and said enlarged image display means are integrally formed as an enlarged display support means. 4. The apparatus according to claim 1, further comprising a state setting unit configured to set a state in which a state in which the enlarged display processing of the specified area is temporarily stopped ignoring a detection result of the line-of-sight detection unit is releasable. The display device according to the above. 5. A display device having a support function for supporting communication of a user's intention, wherein a gaze detecting means for detecting the gaze of the user, and a gaze of the user based on a detection result of the gaze detection means. Area specifying means for specifying an area on the screen of the display means to which the image is directed; recognition means for recognizing an image displayed in the specified area; and specifying the area according to a recognition result of the recognition means. And an enlargement processing unit for selecting an image in the selected area and enlarging the selected image, and an enlarged image display unit arranged at a position visible to the user and displaying the enlarged image. Characteristic display device. 6. The apparatus according to claim 5, wherein the area specifying unit specifies an area in which the user's line of sight is directed for a predetermined time or more based on a detection result of the line of sight detecting unit.
The display device according to the above. 7. The display device according to claim 5, wherein said line-of-sight detection means and said enlarged image display means are integrally formed as enlarged display support means. 8. A state setting means for ignoring a detection result of the line-of-sight detection means and setting a state in which a state of temporarily stopping the enlarged display processing of the image of the specified area is set to be releasable. Item 6. The display device according to Item 5. 9. A display device having a support function for supporting communication of a user's intention, wherein a gaze detecting means for detecting the gaze of the user, and a gaze of the user based on a detection result of the gaze detecting means. Display comprising: an area specifying means for specifying an area on a screen of a display means to which the character is directed; and a voice conversion processing means for converting a character string displayed in the specified area into a voice. apparatus. 10. The detection result of said line-of-sight detection means and the sound generated by said sound conversion processing means so that the detection result of said line-of-sight detection means is ignored and the sound conversion operation by said sound conversion processing means is temporarily stopped. 10. The display device according to claim 9, further comprising independent state setting means for setting a state independent of the conversion operation to be releasable. 11. The display device according to claim 10, wherein the area specifying means specifies an area in which the user's line of sight is directed for a predetermined time or more based on a detection result of the line of sight detecting means. 12. An enlarged display support method for enlarging and displaying an image displayed on a display means according to a user's intention, wherein the step of detecting a line of sight of the user is performed based on a result of the detection of the line of sight. A step of specifying an area on the screen of the display unit to which the user's line of sight is directed; a step of enlarging the specified area at a predetermined magnification; and an enlargement disposed at a position visible to the user. Displaying the enlarged area on an image display means. 13. The enlarged display according to claim 12, further comprising a step of ignoring the detection result of the line of sight and setting a state in which the enlarged display processing of the specified area is temporarily stopped so as to be releasable. How to help. 14. An enlarged display support method for enlarging and displaying an image displayed on a display means according to a user's intention, wherein the step of detecting a line of sight of the user is performed based on a result of the detection of the line of sight. A step of specifying an area on the screen of the display unit on which the user's line of sight is directed; a step of recognizing an image displayed in the specified area; and Selecting an image in the selected area and enlarging the selected image, and displaying the enlarged image on enlarged image display means arranged at a position visible to the user. The enlarged display support method. 15. The method according to claim 14, further comprising a step of ignoring the detection result of the line of sight and setting a state in which the enlarged display processing of the image of the specified area is temporarily stopped so as to be releasable. Enlargement display support method. 16. A speech support method for selecting and characterizing a character string displayed on a display means according to a user's will, wherein the step of detecting the user's line of sight, and the result of detecting the line of sight of the user Specifying a region on the screen of the display means on which the user's line of sight is pointed based on, and converting a character string displayed in the specified region into voice. Speech support method to do. 17. A state in which the detection result of the line of sight and the conversion operation to the sound are independent so that the conversion operation from the character string to the sound is temporarily stopped while ignoring the detection result of the line of sight. 17. The method according to claim 16, further comprising the step of: setting to be able to cancel.