JPH0876968A - Acoustic display device - Google Patents

Abstract

PURPOSE: To provide an acoustic display device capable of searching an object without depending upon a visual sense feedback. CONSTITUTION: The acoustic display device is provided with an interactive system 100 having a graphical user interface, a cursor moving event acquiring part 201 for acquiring a cursor moving event from the system 100, a cursor coordinate acquiring part 202 for acquiring the coordinates of a cursor from the system 100, an object information acquiring part 203 for acquiring the information of an interactive object existing on the cursor coordinates from the system 100, a history managing part 205 for storing the history of the acquired interactive object information, an object-sound correspondence table 206 indicating correspondence between the interactive object information and acoustic information, an object sound controlling part 204 for determining acoustic information by the use of the managing part 205 and the table 206, and a sound generating part 30 for outputting the determined acoustic information as a practical sound.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a user interface of a computer system, and more particularly to a graphical user interface (hereinafter referred to as GU) without visual feedback.
The present invention relates to an acoustic display device capable of utilizing I).

[0002]

2. Description of the Related Art A GUI realizes flexible and wide information presentation by using a two-dimensional information presentation method. However, since the GUI relies on visual information presenting means, it has been difficult for the user to perform operations without always staring at the CRT display.

A synergy FX (SYNAGY / FX) as a dynamic information acoustic display device for expressing dynamic information (state change) in GUI by aural information presentation.
Are known as products (Reference: Synergy)
User's Guide / Reference, pp. 79-89,
ASCII Publishing). Synergy FX
In Y / FX), sound presentation can be performed for, for example, iconizing a window, maximizing a window, starting an application, pressing a button, selecting a menu, and the like. Such a function can be realized by, for example, the apparatus configuration shown in FIG.

That is, when a user operates the interactive system 10, an event occurs in the interactive system 10. However, the event acquisition unit 4 of the dynamic information sound converting unit 40.
01 acquires the event information, and outputs the event sound control unit 4
Enter in 02. The event sound control unit 402 determines acoustic information from the event information input using the event sound correspondence table 403, and the sound producing unit 30 outputs this acoustic information.

[0005]

In the above-described structure, when an event occurs in a dialogue system having a GUI, that is, when a user performs some operation and the state of the dialogue system changes, a sound corresponding to the event is generated. Only the information is output, and the user cannot obtain the static information of the dialogue object before the operation by the acoustic information. That is, the user can confirm the operation by the acoustic display, but cannot search for the interactive object to be operated based on the acoustic display.

SUMMARY OF THE INVENTION An object of the present invention is to provide an acoustic display device which eliminates the above drawbacks of the prior art and acoustically displays the information of a dialogue object existing under the cursor only by moving the cursor on the screen. is there.

[0007]

An acoustic display device according to the present invention is a dialogue system having a GUI, and in the dialogue system, static information for converting static information of a GUI dialogue object existing at cursor coordinates into acoustic information. An information sound conversion unit and a sound generation unit that outputs the sound information determined by the static information sound conversion unit are provided.

The acoustic display device according to the present invention is a dialogue system having a GUI, and in the dialogue system, static information acoustic conversion for converting static information of a GUI dialogue object existing at cursor coordinates into acoustic information. Section, a dynamic information acoustic conversion section for converting a state change of the dialogue system into acoustic information, a sound generation section for outputting acoustic information determined by the static information acoustic conversion section and the dynamic information acoustic conversion section. , Are provided.

In one aspect of the present invention, the static information sound conversion unit includes a cursor movement event acquisition unit that acquires a cursor movement event from the dialogue system, and a cursor that acquires cursor coordinates from the dialogue system. A coordinate acquisition unit, an object information acquisition unit that acquires information of a dialog object existing at the cursor coordinates obtained by the cursor coordinate acquisition unit from the interaction system as object information, and a history management unit that stores a history of the object information. , An object sound correspondence table indicating a correspondence between the object information and the sound information, and a history management unit and an object sound control unit for determining the sound information using the object sound correspondence table.

[0010]

In the present invention, by providing the static information sound converting unit, the information of the dialogue object existing at the cursor coordinates in the dialogue system having the GUI is converted into the acoustic information and presented to the user as the sound. Therefore, the user can search for the interactive object to be operated by moving the cursor. Further, the dynamic information / sound conversion unit converts the event of the dialogue system into sound information and presents the sound to the user. Therefore, the user can acoustically confirm the change in the state of the system.

With the above configuration, the user can perform a series of operations in the GUI of searching for an operation target object → operation → confirming a state change without relying on visual feedback.

[0012]

Next, an embodiment of the present invention will be described with reference to FIGS.

FIG. 1 is a configuration diagram of an acoustic display device according to an embodiment of the present invention, and a dialogue system 1 having a GUI.
0, a static information sound conversion unit 20 that converts static information of the dialog object existing at the cursor coordinates into sound information.
A dynamic information sound conversion unit 40 for converting an event in a dialogue system into sound information, and a static information sound conversion unit 20
And a sound producing unit 30 that outputs the acoustic information determined by the dynamic information acoustic converting unit 40 as an actual sound.

The static information sound conversion unit 20 includes a cursor movement event acquisition unit 201 for acquiring a cursor movement event from the dialogue system 10, a cursor coordinate acquisition unit 202 for acquiring cursor coordinates from the dialogue system 10, and a cursor coordinate acquisition unit. An object information obtaining unit 203 that obtains information of a dialog object existing at the cursor coordinates obtained by the unit 202 from the dialog system 10, and a history management unit 205 that stores a history of the dialog object information obtained by the object information obtaining unit 203. And an object sound correspondence table 206 indicating a correspondence between the dialog object information and the sound information obtained by the object information acquisition unit 203.
And an object sound control unit 204 that determines sound information to be output using the history management unit 205 and the object sound correspondence table 206.

The dynamic information sound converter 40 has the same configuration as the dynamic information sound converter shown in FIG. 6 as the prior art. Therefore, it is indicated using the same reference numbers as in FIG.
It will not be described again.

First, the static information sound conversion unit 20 of the sound display device shown in FIG. 1 will be described in accordance with the processing procedure.

The interactive system 10 having a GUI generates an event in response to various state changes, but the cursor movement event acquisition unit 201 always monitors the cursor movement event, and when the cursor movement event occurs, the sounding trigger is set to the cursor coordinate. Input to the acquisition unit 202. The cursor coordinate acquisition unit 202 uses the cursor movement event acquisition unit 20.
As soon as a sounding trigger is input from 1, a dialogue system 1
The cursor coordinates are obtained from 0 and input to the object information obtaining unit 203. Next, the object information acquisition unit 20
3 uses the cursor coordinates input from the cursor coordinate acquisition unit 202 to acquire the information (for example, type, title name, state, title name of the parent window of the object, etc.) of the interactive object existing at the coordinates, The obtained object information is input to the object sound control unit 204.

Here, since the movement of the cursor occurs continuously with a certain time, the cursor movement event occurs continuously. This means that, for example, when the cursor moves on the same interactive object, the same object information is continuously input to the object sound control unit 204.

Therefore, the history management unit 205 stores the object information input to the object sound control unit 204 in the past at least once. When new object information is input, the object sound control unit 204 compares the latest history information stored in the history management unit 205 with the input object information, and if they are different, the object sound correspondence table 20
6, the sound information corresponding to the object information input from the object information acquisition unit 203 is determined, and the sound information is input to the sound generation unit 30. The sounding unit 30 immediately sounds the input acoustic information.

Examples of the sound generator 30 include a PCM file reproducing device, a voice synthesizing device, and a method using a MIDI sound source.

On the other hand, in the dynamic information sound conversion unit 40, the event acquisition unit 401 acquires event information and inputs the event information to the event sound control unit 402. The event sound control unit 402 determines acoustic information from the input event information using the event sound correspondence table 403, and
0 is output.

FIG. 2 shows an example of the event sound correspondence table 403. FIG. 3 shows an example of the object sound correspondence table 206.

First, the description (example) of the event sound correspondence table 403 and the object sound correspondence table 206 will be briefly described.

The event sound correspondence table 403 defines various events occurring in the dialogue system and acoustic information corresponding thereto. Object sound correspondence table 20
Reference numeral 6 defines the object information obtained by the object information acquisition unit 203 and the corresponding acoustic information.

In the event sound correspondence table 403 and the object sound correspondence table 206, when the output format column is PCM output, the PC output in the output data column
The M file name is described. On the other hand, when the output format field is voice output, the output data field describes a character string to be voice-synthesized. At this time, the part surrounded by "" is a variable. For example, the object sound correspondence table 20 of FIG.
There is a character string "It is a" Title name "menu" in the output data column of 6, but the "Title name" of this character string
Is a variable, in which the title of the object input to the object sound control unit 204 as one of the object information is entered at any time.

The operation of the acoustic display device of FIG. 1 will be described below in accordance with its processing procedure. FIG. 4 and FIG. 5 are image diagrams of usage of the acoustic display device.

In FIG. 4, the arrow cursor 501 is 501.
When the cursor is moved to the position shown in FIG.
Get more cursor movement events. At this time, the event acquisition unit 401 inputs the cursor movement event to the event sound control unit 402.
If there is no cursor movement event in the event column of No. 3 (see FIG. 2), the process is stopped here.

On the other hand, a cursor movement event acquisition unit 201
Inputs a pronunciation trigger to the cursor coordinate acquisition unit 202. The cursor coordinate acquisition unit 202 acquires the cursor coordinates at that time from the interactive system 10 and inputs the cursor coordinates to the object information acquisition unit 203. Object information acquisition unit 20
The object information 3 present at the cursor coordinates obtained by the cursor coordinate acquisition unit 202 is transmitted to the interactive system 10.
Get more. In this case, for example, <type = menu,
Object information such as title name = file, parent window name = memo pad> is obtained, and the information is input to the object sound control unit 204.

The object sound control unit 204 refers to the latest history of the object information stored in the history management unit 205, and if it matches the input object information, the process is terminated. The latest history,
When the input object information does not match, the object sound control unit 204 refers to the object sound correspondence table 206. In this case, the acoustic information column corresponding to the menu of the object column of the object sound correspondence table 206 is <output format = voice output, output data =
"It is the" Title name "menu">. Here, first, <file>, which is the title name of the object information input to the object sound control unit 204, is substituted as a character string into a variable “title name” in the character string in the output data column. As a result, the object sound control unit 204 outputs <output format = voice output, output data = as acoustic information corresponding to the input object information.
"File Menu">
To enter. In this case, the sounding section 30 uses the voice synthesizer to generate a voice "File menu" (see FIG. 4). Based on this sound presentation, the user can search for a dialogue object to be operated.

Further, as shown in FIG. 5, it is assumed that the user actually operates the file menu and opens the menu. At this time, a menu open event occurs in the interactive system 10. Event acquisition unit 401
Acquires this menu open event and inputs it to the event sound control unit 402. The event sound control unit 402 refers to the event sound correspondence table 403 in FIG. 2 and responds to the menu opening by <output format = PCM output, output data = open. The sound information “wav>” is obtained and input to the sound generator 30. The sound generator is op using a PCM output device.
en. Output a wav file. The user can confirm that the menu has actually been opened by listening to this audio presentation (see FIG. 5).

[0031]

According to the present invention, by providing the static information sound conversion unit, the static information of the dialogue object existing at the cursor coordinates can be known by sound presentation. This allows the user to search for a dialog object without relying on visual feedback.

By providing both the static information sound conversion unit and the dynamic information sound conversion unit, object search → operation →
It is possible to perform a series of tasks of confirming state changes without relying on visual feedback.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an acoustic display device in an embodiment.

FIG. 2 is a diagram showing an acoustic display device according to an embodiment,
It is a figure showing an event sound correspondence table.

FIG. 3 is a diagram illustrating an acoustic display device according to an embodiment,
It is a figure which shows an object sound corresponding | compatible table.

FIG. 4 is a diagram showing an acoustic display device according to an embodiment,
It is an image figure about static information sound presentation.

FIG. 5 illustrates an acoustic display device according to an embodiment.
It is an image figure regarding dynamic information sound presentation.

FIG. 6 is a configuration diagram showing a conventional dynamic information acoustic converter.

[Explanation of symbols]

 10 Dialogue System 20 Static Information Acoustic Conversion Unit 30 Sound Generation Unit 40 Dynamic Information Acoustic Conversion Unit 201 Cursor Movement Event Acquisition Unit 202 Cursor Coordinate Acquisition Unit 203 Object Information Acquisition Unit 204 Object Sound Control Unit 205 History Management Unit 206 Object Sound Correspondence Table 401 Event acquisition unit 402 Event sound control unit 403 Event sound correspondence table 501 Cursor

Claims (4)

[Claims] 1. A dialogue system having a graphical user interface, and in the dialogue system, static information acoustic conversion for converting static information of a dialogue object of the graphical user interface existing at cursor coordinates into acoustic information. An acoustic display device comprising: a unit; and a sounding unit that outputs the acoustic information determined by the static information acoustic conversion unit. 2. A dialogue system having a graphical user interface, and in the dialogue system, static information acoustic conversion for converting static information of dialogue object of the graphical user interface existing at cursor coordinates into acoustic information. A dynamic information acoustic conversion unit for converting a change in the state of the dialogue system into acoustic information, and a sound generation unit outputting the acoustic information determined by the static information acoustic conversion unit and the dynamic information acoustic conversion unit. An acoustic display device comprising: 3. The static information sound conversion unit, a cursor movement event acquisition unit that acquires a cursor movement event from the dialog system, a cursor coordinate acquisition unit that acquires cursor coordinates from the dialog system, and the cursor coordinates. An object information acquisition unit that acquires information of the interactive object existing at the cursor coordinates obtained by the acquisition unit as the object information from the interactive system, a history management unit that stores a history of the object information, and the object information and the acoustic information. An object sound correspondence table indicating correspondence, an object sound control unit that determines acoustic information using the history management unit and the object sound correspondence table,
The acoustic display device according to claim 1 or 2, comprising: 4. The dynamic information / acoustic converter acquires an event acquisition unit that acquires event information from the dialogue system, an event sound correspondence table indicating correspondence between the event information and the sound information, and the event acquisition unit. The sound display device according to claim 3, further comprising: an event sound control unit that determines sound information from the acquired event information using the event sound correspondence table.