JPH08185255A - Screen operation device - Google Patents

Abstract

(57) [Abstract] [Purpose] In order to perform multiple operations simultaneously on the operation screen of a computer, etc., the operability is degraded by the operation of only the hand. Therefore, the purpose is to automatically operate the screen using a part of the body. In order to achieve this object, a screen operating method according to the present invention comprises an infrared light emitting section having an infrared light emitting source for detecting movement of a part of a body, and the infrared light emitting section. A position detecting unit for detecting infrared light emitted from the light emitting unit, and a calculating unit for allocating screen operation information from the position detecting unit and performing automatic operation are provided. According to the present invention, since the position detecting unit determines the movement of a part of the body by the infrared light emitting unit having the infrared light emitting source built-in, for example, by a simple motion of moving the neck. , It becomes possible to automatically operate the specific operation contents arranged in advance, which improves work efficiency.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a screen operating device such as a computer.

[0002]

2. Description of the Related Art In recent years, computers have been operated such that modes such as operating a plurality of external devices or operating a plurality of programs at the same time can be displayed. The programs can be selected interactively or detailed graphics can be displayed. It is designed to be easy to understand using functions.

In recent years, in a display device for a computer, a mode display for simultaneously operating a plurality of external devices or a plurality of programs on the screen has been performed. For example, there is a usage mode in which an operation screen for operating a computer is displayed on a display display device, a TV broadcast screen is displayed at the same time, an operation screen of another connected device is displayed, and multiple types of screens are displayed at once. It is rising.

It is very convenient to control a number of external devices at the same time, and the intention is to simultaneously display a plurality of types of operation screens on the same screen.

A conventional screen operation device will be described below. FIG. 12 shows a conventional screen operation device.

First, the computer operation I shown in (1) is performed to display the screen I. Next, the computer operation II shown in (2) is performed midway to display the screen II. Then, the first operated screen I may be hidden by the screen II and may not be visible. That is, as a result of the computer uniquely deciding the layout and size of the screen display, when the operator tries to return to the computer operation I again, the cursor is moved to the position of the screen I which was initially present and the screen II is changed to the screen I. The operator had to change the display each time, that is, the operation of excluding it from above.

[0007]

However, in the above-mentioned prior art, even if the computer is operated interactively or the graphic function is enhanced to display the operation method in an easy-to-understand manner, the human can operate the finger at a time. There is a limit to what you can do. Moreover, in the interactive form, there is a growing expectation that in addition to the conventional finger button operation, it is desired to perform a gesture operation. In other words, the expectation is to operate the computer as usual, while moving the arms and the head to make the conversation more like a human conversation.

The present invention solves the above-mentioned problems.
For any movement intended by the operator using any part of the body,
To provide an operation method to operate automatically.

Alternatively, with respect to a plurality of movements intended by the operator using an arbitrary part of the body, the operation content is automatically discriminated from each movement, and each specific operation is automatically performed. The purpose is to provide.

In addition, the operator changes the display of the screen each time, that is, each time an inconvenience occurs, the size and the display position of the operation screen are changed so that the operator can easily operate the screen. Had a problem of decreasing.

The claims of the present invention solve the above-mentioned problems, and provide an apparatus for automatically performing an arbitrarily determined operation by moving the neck to an arbitrary position intended by the operator. The purpose is to

[0012]

In order to achieve this object, the present invention provides an infrared light emitting section containing an infrared light emitting source for detecting movement of a part of a body, and the infrared light emitting section. The light emitting unit includes a position detecting unit for detecting infrared light emitted from the light emitting unit, and a calculating unit for assigning operation contents from the position detecting unit.

Alternatively, a plurality of operating infrared light emitting portions for detecting movement of a part of the body, and a plurality of operating operations for detecting infrared light emitted from the plurality of operating infrared light emitting portions. The operation position detecting section and the plural operation calculating section to which the operation content is assigned from the plural operation position detecting section.

In addition, in order to detect the infrared light emitted from the head infrared light emitting section and the head infrared light emitting section which incorporates an infrared light emitting source for detecting the direction of a part of the body. The visual line position detection unit and the calculation unit that allocates the screen operation content from the visual line position detection unit.

[0015]

With this configuration, the infrared light emitting section containing the infrared light source for detecting the movement of a part of the body can be fixed to a part of the body, for example, the head. Next, the position detection unit is directed to the infrared light emitting source in order to receive the infrared light and detect the movement of a part of the body. For example, when fixing the infrared light emission source to the head, the infrared light from the infrared light emission source is fixed to the head so that it emits toward the operation screen, and the position detection unit displays the infrared light on the side of the operation screen. It is arranged facing the light emitting source. The movement of a part of the body at this time, for example, the movement of the head, is detected by the operator looking at the screen because the direction of the infrared light fixed to the head matches the direction of the head. It is possible to determine whether or not work is being performed by following the movement of the infrared light emission source.

Here, the mechanism for following the movement of the infrared light emitting source means that the portion which collects the infrared light and receives and detects the infrared light is incident from the front by applying the infrared light. The determination is made by having a mechanism that can determine the position of the infrared light source (see Japanese Patent Application No. 2-183698).

That is, by detecting the incident angle of the infrared light emitted from the infrared light emitting source, the position detecting section detects, for example, the direction of the head to which the infrared light emitting source is fixed. , If the operator intentionally makes a certain motion, for example, a motion of laterally shaking the head, from the lateral motion information of the head detected by the position detection unit, when the head registered in advance in the calculation unit laterally moves, In order to operate, the operator performs an operation registered in advance.

Further, the plural-operation infrared light emitting section containing the infrared light source for detecting the movement of a part of the body can be fixed to a part of the body, for example, the head. The infrared light emitted here emits a plurality of different types of infrared light, for example, infrared light having different oscillation frequencies. Next, the multi-operation position detector heads toward the multi-operation infrared light emission source to receive infrared light and detect movement of a part of the body. For example, when fixing the infrared light source for multiple operations to the head, the infrared light from the infrared light source for multiple operations is fixed to the head so as to emit toward the operation screen. Is placed beside the operation screen. The movement of a part of the body at this time, for example, the movement of the head, is detected because the direction of the infrared light fixed to the head and the direction of the head are the same. It is possible to determine whether or not the user is working while moving the robot by following the movement of infrared light of the infrared light emitting sources for multiple operations.

The mechanism for following the movement of the light source of infrared light is the infrared light incident from the front by applying infrared light to the part that collects the infrared light and receives and detects the infrared light. Judgment by having a mechanism that can determine the position of the light source (Japanese Patent Application No. 2-1
83698).

That is, by detecting the incident angles of a plurality of different types of infrared light emitted by the plural operating infrared light emitting sources, for example, the head of the infrared light emitting source is fixed. If the operator intentionally makes a certain movement, for example, shakes his / her head sideways while emitting a certain type of infrared light, the movement of the head by the multiple movement movement detection unit is detected. From the information, a specific lateral movement of the head registered in advance in the calculation unit is detected, and the pre-registered operation intended by the operator is automatically performed. For example, if another head movement is performed by vertically swinging the head while emitting another type of infrared light, the head movement information from the plural movement motion detection unit is used to notify the plural movement calculation unit in advance. With another movement of the same head registered, another pre-registered operation intended by the operator is automatically performed.

Further, with this configuration, the head infrared light emitting section having a built-in infrared light emitting source for detecting a part of the body, for example, the direction in which the head faces, can be fixed to the head.
The infrared light emission source is for emitting infrared light toward the screen being operated, and is fixed to the head so that the infrared light from the infrared light emission source emits toward the operation screen. To do.
The operator is working while looking at the screen,
As the operator is looking at what the operator is looking at, the direction of the infrared light fixed to the head matches the direction of the head. The coordinates of the upper viewpoint can be easily determined. Next, the line-of-sight position detection unit for detecting the infrared light emitted from the head infrared light emitting unit, by applying the infrared light to the portion that collects the infrared light and receives and detects the infrared light, It has a mechanism that can identify the position of the light source of infrared light incident from the front (Japanese Patent Application No. 2-183698).
(See the official gazette). In other words, it is a part that detects the incident angle of infrared light to detect the direction in which the head to which the light source of infrared light is fixed is facing, and indirectly detects the direction of the line of sight. The calculation unit that allocates the screen operation content based on the information from the line-of-sight position detection unit uses the information of the position of the line-of-sight on the screen of the operator determined from the incident angle of infrared light to determine the arbitrary operation content that has been previously allocated. To execute. That is, the operator wearing the head infrared light emitting section operates, for example, toward the operation screen displayed on the display display tube or the like. At this time, a plurality of types of operation screens are displayed on the screen. At this time, the line-of-sight position detection unit installed beside the display tube receives the infrared light emitted from the head infrared light emitting unit and detects the position of the light emitting source. The detected position information of the light emitting source is recalculated to the position information of the line of sight of the operator on the screen. If the operator moves a line of sight to the specific position at an arbitrary position on the screen in which the operator is operating in advance, if a certain specific operation is performed, the specific operation is automatically performed only by moving the line of sight. It will be executed.

[0022]

【Example】

(First Embodiment) A first embodiment of the present invention will be described below with reference to the drawings.

In FIGS. 1 and 2, reference numeral 1 denotes an infrared light emitting portion, which has an infrared light emitting source built-in for detecting the movement of a part of the body. The infrared light emitting section 1 can be fixed to a part of the body, for example, the head. The mounting method is such that infrared light is emitted toward the display screen of the display display tube being operated. The purpose of fixing the infrared light emitting unit 1 to the head and emitting infrared light toward the operation screen is that the operator is working while looking at the screen. This is because the movement of the head, which is a part of, is coordinated and operated. Next, 2 is a position detecting section for detecting the infrared light emitted from the infrared light emitting section 1. The function of the position detector 2 will be described in detail with reference to FIG. Reference numeral 4 denotes an optical diaphragm for transmitting infrared light sent from the infrared light emitting section 1 only in a portion having a minimum area. Reference numerals 5a, 5b, 5c and 5d denote infrared light receiving elements, which are arranged immediately behind the optical diaphragm 4. Here, for example, the four-divided infrared light receiving elements 5a to 5d combined in a square shape are used. A switching circuit 6 is a circuit that selects one or more of the four infrared light receiving elements 5a to 5d and sends a light receiving signal to the next stage. Reference numeral 7 denotes a received light intensity detection unit, which is connected to the output of the switching circuit 6 and quantitatively detects the intensity of light received by the infrared light receiving elements 5a to 5d. Reference numeral 8 denotes a calculation unit, which is a unit that performs calculation processing on the output result from the received light intensity detection unit 7.

That is, the infrared light sent from the infrared light emitting portion 1 located at a position several meters away from the front of the optical diaphragm 4 is transmitted by the optical diaphragm 4 only in a portion having a very small area, and the infrared light is transmitted. It corresponds to the light receiving elements 5a to 5d. Infrared light receiving element 5a
The light intensity received by ~ 5d is processed into a quantitative value.
Of the light intensity of the infrared light receiving elements 5a, 5b, 5c, and 5d, if the position 5a and the upper left position in the arrangement shown in FIG. The light emitting section 1 is located in the lower left of the front when viewed from the infrared light receiving elements 5a to 5d. In the case of the infrared light receiving element 5b, the lower right of the front, the infrared light receiving element 5c, the upper right of the front, and the infrared light receiving element 5d, the upper left of the front. By using this principle, the position detection units 2 arranged at four places around the display screen receive infrared light emitted from the infrared light emitting unit 1 toward the screen from the operator in front of the screen, respectively. .

Next, the position detection of the infrared light emitting section 1 located in front of the position detection section 2 at four positions will be described in detail. The case 1 in FIG. 4 is a case where infrared light falls on the screen. The measurement result of the light intensity at each position is shown from the measurement data of the position detecting section 2 at four positions. The neck direction of the operator in the screen direction, which is comprehensively determined, is the upper left area in the screen. In the case of 1 ', it was determined that the operator's comprehensively determined neck direction in the screen direction was the upper left area in the screen. In this way, although roughly, the direction of the neck in the screen direction is inferred from the position of the light emitting source of infrared light. Here, it is assumed that the head is swung to the left and right from the two positions of the neck in the screen direction. In other words, it goes back and forth between the upper left area in the screen and the left and right areas in the screen.

In FIG. 1, reference numeral 3 denotes a calculation section, which is a section for judging and executing an operation desired by the operator based on the neck orientation information in the screen direction. For example, the left and right movements of the neck, which is a part of the body registered in advance, are made to correspond to the operation contents registered by the operator. Suppose you select a certain option while operating the screen of a computer. At this time, if the operator shakes his / her head to the left and right, if the operation content is registered as "No", the user can operate the computer simply by shaking the head to the left or right.

Although only one infrared light emitting section 1 is attached to the operator's head, the number can be set arbitrarily.
Further, the mounting position is not limited to the right side of the head and the right ear.
Although the position detection unit 2 has shown an example of installation at four locations adjacent to the periphery of the operation screen, the installation positions and the number of installations do not have to be adjacent, and the number can be set arbitrarily. The movement of a part of the body that is registered in advance is not limited to shaking the head from side to side.

Although infrared light is used in the present application, it is not limited to infrared light as long as it is a means for transmitting a signal.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings.

In FIG. 5, reference numeral 9 denotes an infrared light emitting section for a plurality of operations, which incorporates an infrared light emitting source for detecting the movement of a part of the body. Then, the plural-operation infrared light emitting section 9 emits plural kinds of infrared light of different types, for example, infrared light having different transmission frequencies. The plural-operation infrared light emitting section 9 can be fixed to a part of the body, for example, the head. The mounting method is such that infrared light is emitted toward the display screen of the display display tube being operated. The purpose of fixing the plural-operation infrared light emitting section 9 to the head and emitting infrared light toward the operation screen is that the operator is working while looking at the screen. And to move the head, which is a part of the body, in cooperation with each other. Next, reference numeral 10 denotes a plural operation position detecting section for detecting the infrared light emitted from the infrared light emitting section 9 for plural operations. Multiple motion position detector 1
The function of 0 is the same as the operation content of the position detecting unit 2 described in detail in FIGS. 3 and 4 in the first embodiment.
Next, for example, a plurality of movement position detection units 1 arranged at four locations
The position detection of the plural operation infrared light emitting sections 9 located in front of the detection result of 0 will be described in detail.

In FIG. 6 (A), the infrared light emitting section 9 for plural operations emits an infrared light 11A of a certain kind.
Is on the screen and is shaking your head to the left and right.
In FIG. 6 (B), in the plural operation infrared light emitting section 9,
This is a case where another type of infrared light 11B hits the screen and the head is shaken up and down. As described above, the method of estimating the neck direction in the screen direction from the position of the light emitting source of infrared light is roughly as described in detail in FIG. 4 in the first embodiment. Here, it is assumed that the head is shaken left and right and the neck is shaken up and down based on two kinds of movements of the head in the screen direction. It is assumed that the two types of infrared light emission methods are alternately performed three times per second, for example.

Numeral 12 is a calculation unit for a plurality of movements, which shows only the neck direction information of the infrared light 11A in the screen direction from the operation status of the screen.
Alternatively, it is a part for judging and executing an operation desired by the operator based on only the neck direction information of the screen direction of the infrared light 11B.
For example, the operator registers in advance that the infrared light 11A reacts only to left and right movements and the infrared light 11B responds only to vertical movements. When the head is shaken to the left and right, the left and right neck movements registered in the plural movement calculation unit are defined as "no", that is, no operation is performed, and the operation is not automatically performed. When the head is shaken up and down, the movements of the upper and lower necks registered in the calculation unit for a plurality of movements are “Yes”, that is, if the operation is defined as an operation, the operation is automatically advanced.

In addition, according to the operation situation, the infrared light 11
If the computer determines that only A is operated, it is possible to react the neck to vertical and horizontal movements only with the infrared light 11A.

Here, it is assumed that only one infrared ray emitting section 9 for plural operations is attached to the operator's head, but the number can be set arbitrarily. Further, the mounting position is not limited to the right side of the head and the right ear. The position detector 10 for plural movements shows four installation examples adjacent to the periphery of the operation screen.
The numbers of installation are not necessarily adjacent to each other, and the number can be set arbitrarily. The movement of a part of the body that is registered in advance is not limited to shaking the head up and down and left and right. The plurality of infrared lights is not limited to two, and the transmission information is not always differentiated by changing the frequency. Further, the means for transmitting signals is not limited to infrared rays.

(Embodiment 3) An embodiment of the present invention will be described below with reference to the drawings.

In FIGS. 7 and 8, reference numeral 101 denotes a head infrared light emitting portion, which incorporates an infrared light emitting source for detecting head movement. The infrared light emitting unit 101 can be fixed to the head. It is for emitting infrared light toward the display screen of the display tube which is being operated, for example. By fixing the infrared light emitting unit 101 to the head,
The purpose of emitting infrared light toward the operation screen is that the operator is working while looking at the screen. Since the movement of the external light and the movement of the head match, the movement of the infrared light emission source is tracked, so that the coordinates of the viewpoint on the screen can be easily detected from the movement of the head on the head. It is something that is fixed. Next, a line-of-sight position detection unit 102 is for detecting the infrared light emitted by the head infrared light emitting unit 101. The function of the line-of-sight position detection unit 102 will be described with reference to FIG. Reference numeral 104 denotes an optical diaphragm that transmits infrared light sent from the head infrared light emitting unit 101 only in a portion having a minimum area. 105a, 105b, 105
Reference numerals c and 105d denote infrared light receiving elements, which are arranged immediately behind the optical diaphragm 4. Here, for example, an infrared light receiving element 105a to 105d divided into four is combined in a square shape. A switching circuit 106 is a circuit that selects one or more of the four infrared light receiving elements 105a to 105d and sends a light receiving signal to the next stage. Reference numeral 107 denotes a received light intensity detection unit, which is connected to the output of the switching circuit 106 and quantitatively detects the intensity of light received by the infrared light receiving elements 105a to 105d. 108 is a calculation unit,
This is a part that performs arithmetic processing on the output result from the received light intensity detection unit 107. In other words, the infrared light sent from the head infrared light emitting unit 101 located at a position several meters away from the front of the optical diaphragm 104 is transmitted by the optical diaphragm 104 only in a portion having a very small area, and the infrared light is received. It corresponds to the elements 105a to 105d. The intensity of light received by the infrared light receiving elements 105a to 105d is processed into a quantitative numerical value. Now, the infrared light receiving element 10
Of the light intensities of 5a, 105b, 105c, and 105d, if the position of 5a and the upper left position in the arrangement shown in FIG. 9 receive the strongest light, the head infrared light emitting unit 1 emits red light. When viewed from the external light receiving elements 5a to 5d, it is in the front lower left direction. In the case of the infrared light receiving element 5b, it is located in the lower right front. In the case of the infrared light receiving element 5c, the upper right corner. Infrared light receiving element 5d
If so, it will be the upper left corner. By using this principle, the head movement detection units 2 arranged at four positions around the display screen are provided with infrared light emitted by the head infrared light emitting unit 1 toward the screen from the operator in front of the screen, respectively. To receive.
Next, a detailed description will be given of the position detection of the head infrared light emitting unit 1 located in the front from the detection results of the four line-of-sight position detecting units 2. The case of 1 in FIG. 10 is a case where the screen is exposed to infrared light. The measurement results of the light intensity at each position are shown based on the measurement data of the gaze position detecting units 2 at four positions. The operator's in-screen gaze position determined comprehensively is the upper left area in the screen. In the case of 1 ', the operator's in-screen gaze position determined comprehensively is determined to be the lower left area in the screen. As described above, the line-of-sight position on the screen is inferred from the position of the light emitting source of infrared light, though it is roughly. Three
Is a part for calculating and executing an operation desired by the operator based on the line-of-sight position information on the screen. For example, it is a part for displaying the operation screen layout registered by the operator. This display will be described in detail with reference to FIG. Figure 11
11 (1), the operation I, I ′, I ″ is performed. Next, in FIG. 11 (2), the operation II is performed. Then, the operator sends a line of sight to the display screen I. At this time, the line-of-sight position detection unit 102 installed beside the display tube receives the infrared light emitted from the head infrared light emitting unit 101, and The position of the light emitting source is detected and it is determined that the position of the line of sight of the operator on the screen is in the area of the operation I. The calculating unit 103 obtains the line of sight position information from the line of sight position detecting unit 102, and It is determined that the operation I is about to return to the operation I. At this time, the operator records that the operation I is arranged so that the operation I can be operated when the line of sight is sent to the operation I region. When the operator tries to perform operation II, the operation screen operation II Sending gaze, the screen display of the operation II which is registered by the calculating unit 103, to be displayed on the display the display tube screen.

Here, it is assumed that only one head infrared light emitting section 101 is mounted on the head of the operator, but the number can be set arbitrarily. Further, the mounting position is not limited to the right side of the head and the right ear. Although the line-of-sight position detection unit 102 is shown as being installed at four locations adjacent to the periphery of the operation screen, the installation positions and the number of installations need not be adjacent, and the number can be set arbitrarily.
Further, not only the line of sight, but any instruction such as pointing with a finger and recognizing the instruction may be used.

[0038]

INDUSTRIAL APPLICABILITY As described above, according to the present invention, an infrared light emitting section having a built-in infrared light emitting source for detecting movement of a part of a body and infrared light emitted by the infrared light emitting section are provided. The position detection unit for detecting and the operation unit that recognizes the operation contents from the position detection unit, and the calculation unit that performs the automatic operation provides a simple operation of moving a part of the body to display the specific operation contents agreed in advance. It will be possible to work automatically, improving work efficiency.

Further, a plurality of operation position detectors incorporating an infrared light emitting source for detecting the movement of a part of the body and a plurality of infrared lights emitted from the plurality of operation infrared light emitting portions are detected. By recognizing the operation content from the multiple motion position detection unit and the multiple motion position detection unit for providing a multiple motion calculation unit that performs multiple automatic operations, a simple motion of moving a part of the body, It is possible to operate multiple automated operations, which improves work efficiency.

Further, as described above, according to the present invention, the head infrared light emitting section and the head infrared light emitting section emitting the infrared light emitting section for detecting the direction in which the head faces are emitted. A simple operation of moving the neck by providing a line-of-sight position detection unit for detecting infrared light and a calculation unit that allocates screen operation contents from the line-of-sight position detection unit automatically performs specific operation contents agreed in advance. It becomes possible to work, and work efficiency is raised.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a main part of a first embodiment of the present invention.

FIG. 2 is a diagram showing a main part operation state of the first embodiment of the present invention.

FIG. 3 is a diagram showing an enlarged configuration of a main part of the first embodiment of the present invention.

FIG. 4 is a diagram showing an operating state of a main part of the first embodiment of the present invention.

FIG. 5 is a diagram showing a main configuration of a second embodiment of the present invention.

FIG. 6 is a diagram showing a main part operation state of a second embodiment of the present invention.

FIG. 7 is a diagram showing a configuration of a main part of a third embodiment of the present invention.

FIG. 8 is a view showing a main part operation state of a third embodiment of the present invention.

FIG. 9 is a diagram showing an enlarged configuration of a main part of a third embodiment of the present invention.

FIG. 10 is a diagram showing an operating state of essential parts of a third embodiment of the present invention.

FIG. 11 is a view showing a main part operation state of the third embodiment of the present invention.

FIG. 12 is a diagram showing an operation state of a conventional example.

[Explanation of symbols]

 1 Infrared Light Emitting Unit 2 Position Detecting Unit 3,103 Calculating Unit 101 Head Infrared Light Emitting Unit 102 Gaze Position Detecting Unit

Claims (5)

[Claims] 1. A signal generating means fixed to a part of the body for detecting the movement of the part of the body, a position detecting means for detecting a signal of the signal generating means, and a signal from the position detecting means. A screen operation device comprising a calculation unit for allocating operation contents according to. 2. The screen operating device according to claim 1, wherein the signal generating means generates a plurality of signals. 3. The screen operating device according to claim 1, wherein the signal generating device generates infrared light. 4. A signal generating means fixed to a part of the body for detecting the orientation of the part of the body, a detecting means for detecting a signal generated by the signal generating means, and the detecting means. A screen operation device comprising a calculation unit for calculating a screen operation. 5. The screen operating device according to claim 1, wherein the signal generating means generates infrared rays.