JPH0926850A - User interface device - Google Patents

Abstract

(57) [Abstract] [Purpose] To improve the operability of the graphical user interface using a pointing device such as a mouse, trackball, or tablet. [Configuration] Host computer 110 and mouse 11
In the system consisting of four, the mouse 114 has a reaction amount control unit 1 and a button detection unit 116 as in the past, and a reaction force control unit 1 that applies a reaction force to the user's hand with respect to the movement of the mouse. In the operating environment using the interface, when the pointer operated by the pointing device passes or approaches the object or its vicinity, the information on the operation of the pointing device is fed back to the user via the reaction force control unit 1, and the host Computer 11
At 0, the operation information of the user generated in response to the feedback of the reaction force control unit 1 is processed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a user interface device, and more particularly to a user interface device such as a mouse, tablet or trackball.

[0002]

2. Description of the Related Art Conventionally, input devices such as a keyboard, a tablet, a mouse and a trackball have been used to operate a computer and input data.
In particular, a tablet or a mouse that can input coordinate information and trajectory information to a computer while observing a monitor is often used with a graphical user interface, which can improve the operability of the computer.

In particular, the mouse has a close relationship with the so-called graphical user interface system of recent years, and an intuitive operation is possible by clicking and dragging an icon or a window, which is widely accepted by general users.

FIG. 12 is an external view of a conventional mouse. Reference numeral 100 is a mouse housing, 101 and 102 are mouse buttons operated by a user, and 103 is a lead for receiving data input / output and power supply from a host computer. It is a line.

FIG. 13 shows the side surface of FIG. 12 and FIG. 14 shows the bottom surface of FIG. 12, and reference numeral 104 denotes a spherical detection ball for detecting the position of the mouse. As shown, the detection ball 104 projects from the opening 104a on the bottom surface of the mouse, and is supported by a support mechanism (not shown) so that the detection ball 104 can freely rotate depending on the direction and amount of movement of the mouse. Reference numerals 105 and 106 denote sliding members for stabilizing the sliding of the mouse, which are made of resin members such as polyacetal and Teflon.

FIG. 15 shows the entire system using the mouse 114. Here, reference numeral 110 is a host computer, which is an operating system (OS) 111, an OS.
Alternatively, a mouse driver 11 implemented at the application level for controlling input / output to / from the mouse
2. It includes a mouse interface 113 for hardware connection with a mouse, and is configured to execute various application software.

In FIG. 15, reference numeral 114 is a mouse,
It has a movement amount detection unit 115 and a button detection unit 116 that detects the ON / OFF state of the mouse buttons 101 and 102.

FIG. 16 shows a typical method of a mouse movement amount detecting section. As shown in the figure, inside the mouse, the detection ball 104 has a detection roller 12 that rotates in one axial direction and has an optical disk-shaped slit.
0 and 121 are in contact. Detection rollers 120, 121
For detecting the rotation of the encoder plates 120a and 121a, the components of the movement of the mouse on the placement plane in the respective directions of the two orthogonal coordinate axes are detected by the detection sensors 122 and 123. Reference numeral 124 is a pressure roller for generating a frictional force on the detection roller.

Here, a configuration has been shown in which the mechanical movement of the ball is detected by an optical encoder to detect the amount of movement. In addition to this, a method of directly reading the pattern of the mouse placement plane optically as a detection means, Various methods have been proposed, such as those using magnetic changes.

[0010]

The conventional mouse as described above is a means of a user interface and is used to convey the user's intention to the host computer.

That is, the conventional mouse is one-way information input from the user to the computer, and a display or the like is indispensable for feeding back the input information to the user. There's a problem.

1) When operating an object on a monitor such as dragging an icon or a window (moving the mouse while the mouse button is pressed and pulling an object such as an icon or window on the display). There is no other confirmation method than looking at the display.

2) When selecting an icon or window, it is necessary to place the pointer exactly on the object.

Such a problem also occurs when an attempt is made to configure a similar graphical user interface with a pointing device such as a mouse, trackball or tablet.

An object of the present invention is to solve the above problems and provide a user interface device which can greatly improve the operability when a graphical user interface is constructed using a pointing device such as a mouse, a trackball or a tablet. It is in.

[0016]

In order to solve the above problems, in the present invention, a reaction force to the pointing device with respect to the operation of the pointing device by the user,
Operate with a pointing device in an operating environment that uses a feedback means that feeds back information to the user by mechanical action such as vibration or shock, and a graphical user interface that arranges objects such as icons, windows, and pop-up menus on the display. When the pointer to be passed or approaches an object to be operated or its vicinity, the information about the operation of the pointing device is fed back to the user via the feedback means, and the pointing device of the user according to the feedback. The configuration provided with the control means for processing the information regarding the operation of was adopted.

[0017]

According to the above construction, the control for feeding back the information on the operation of the pointing device to the user through the feedback means and for processing the information on the operation of the pointing device by the user according to the feedback. The provision of the means enables the user interface through means other than the visual display.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail based on embodiments shown in the drawings.

According to the present invention, when the user operates the pointing device, a mechanical action such as vibration, reaction force, and impact is generated to control the response and feel of the pointing device. The operation of the pointing device or the information input from the pointing device is fed back to the user. In the following, a mouse will be mainly shown as an example.

<First Embodiment> FIG. 1 shows the structure of a computer system capable of performing such operations.
The difference from the conventional example is that the mouse 114 is provided with the reaction force control unit 1.

FIG. 2 shows the structure of the mouse 114. Here, as shown in the figure, the reaction force control unit 1 including the members 10 to 14 is provided in the mouse.

The reaction force control section 1 is rotatably supported on a rotation center 12 to support the friction pad 10 and the friction pad 10 having an appropriate friction coefficient for sliding and braking the detection ball 104. The arm 11 includes a fixed arm 11, a magnet 13 fixed to the tip of the arm 11, and an electromagnetic coil 14 that generates a magnetic field at a current value arranged in the mouse so as to oppose the magnet 13.

The reaction force control unit 1 includes an electromagnetic coil 14 and a magnet 1.
The frictional pad 10 and the detection ball 1 for detecting the movement of the mouse with the center of rotation 12 as the fulcrum by the attraction force with the friction pad 3.
A pressure is created during 04. The frictional force generated at that time generates a reaction force.

The strength of the reaction force at this time can be controlled by the value of the current passed through the electromagnetic coil. Further, in order to release the control of the reaction force control unit 1, a spring or the like is urged in advance, or a current of the electromagnetic coil 14 is caused to flow in a direction in which the electromagnetic coil 14 and the magnet 13 repel each other.

At this time, the means for controlling the reaction force is not limited to the method using the electromagnetic coil and the magnet, and any electromechanical conversion element such as a piezoelectric element or a piezo element may be used.

The reaction force control unit 1 includes a host computer 11
It can be appropriately operated by control from 0, and can be used to feed back information from the mouse operation or mouse to the user. Therefore, a signal line for controlling the reaction force control unit from the host computer is added to the mouse interface.

FIG. 3 is an example of a graphical user interface operating in this embodiment. FIG. 3 is a diagram showing a screen of a monitor television connected to a host computer, and reference numeral 30 is an outer frame of the outermost boundary of the window, 3
1 is the title bar of the window, 32 is the menu bar,
33 is a window, 34a, 34b, 34c, 34d and 34e are icons which are objects for virtually displaying a disk file or an input / output device to the user,
Reference numeral 35 is a mouse pointer.

As described above, in an interface using objects such as icons, windows, and menus,
Move, copy, delete files, input and output with peripherals,
Also, move objects such as icons, windows, and menus associated with directories on the file system or application software to make room on the display for windows on your desktop. There is something to do.

The mouse of this embodiment can be used for the following purposes in such a moving operation.

1) A reaction force is generated when the pointer passes over the icon or the window. For example, in FIG. 3, the user moves the menu bar 32, the window 33, and the icons 34a to 34e by moving the position of the pointer 35 with the mouse, and selects and operates with the mouse button 101 or 102 to move the pointer.

At this time, for example, the icon 34b in the figure is displayed.
When the pointer 35 is moved in order to select with the pointer 35, if the pointer is on the icon, the reaction force control unit 1 is operated, and the feeling of use of the mouse during operation is made heavier than usual, so that Make the pointer recognize that the icon can be selected.

Similar reaction force control can be performed when dragging not only the icon but also another object such as the window 33.

2) It may be considered that a reaction force is generated when passing the vicinity of a small object which is difficult to recognize or select the display on the screen. For example, in an image processing application, the user operates the mouse in the window 33 as shown in FIG.
A vertical line 62 may be drawn from the line 61 to the line 61.

At this time, for example, when selecting a point 60 in the figure and drawing a line, this point 60 is extremely small and difficult to recognize with respect to other objects, so that it is within a predetermined distance in the vicinity of this point 60. At a certain time, the reaction force control unit 1 is operated,
If the mouse button 101 or 102 is pressed during that time, it is considered that this point 60 is recognized. As in this example, the reaction force control can be similarly performed not only on points but also on small objects such as lines that are difficult to recognize.

In this way, by the reaction force generated by the mouse, it is possible to give feedback to facilitate the operation of a small object which is difficult to recognize or select the display on the screen.

Whether or not the feedback due to the generation of such a reaction force is performed can be determined in advance depending on the size of the object. For example, when the area of an object or the number of dots in the horizontal or vertical direction forming the object is less than or equal to a certain value, it is possible to generate a reaction force for the object. As a result, the reaction force control can be performed only on an object smaller than a certain size or thickness.

3) It is also possible to feed back the operating state of the menu by generating a reaction force when the menu is inverted.

For example, in the graphical user interface as shown in FIG. 5, a menu selection may be made by dragging an entry on the menu bar 32. In FIG. 5, the pointer 3 operated by the mouse
An item in the pull-down menu as shown in the figure is selected by dragging (or clicking) a certain menu (here, "edit") according to 5. While dragging, cut, copy, paste, cl on the menu
As the entry passes through the ear, the inversion position of each entry moves in sequence, and the item is selected by releasing the depression of the mouse button at the desired position.

In such an environment, there may be an entry that cannot be selected in the pull-down menu depending on the operation mode of the application to which an input / output device is not connected. The user is made aware of it by such means as darkening the display of possible entries.

When operating the pull-down menu in which selectable and non-selectable items coexist, it is conceivable that the reaction force control unit 1 of the mouse applies a reaction force when the entry is selectable.

In this way, it is possible to make the user recognize the unselectable entry in the pull-down menu more reliably than the conventional display-only feedback.

In this way, when the pointer is located at or near an object in a graphical user interface environment for operating an object such as an icon or a window with a mouse having a reaction force control unit as shown in FIGS. Alternatively, by generating a reaction force in the mouse when the pull-down operation is possible, information other than visual information can be fed back to the user.

That is, based on the position information of the mouse (or of the pointer operated by the mouse), when the pointer is in the object or in the vicinity thereof or in a pull-down operable position, the operation / input information is controlled by reaction force control. By providing feedback, it is possible to give feedback to the user about the mouse operation and the input information from the mouse, which assists the operation of the mouse and the feedback of information from the mouse, which has been done only through the conventional display. The burden on the person can be reduced.

Further, the reaction force may not be generated for an object that cannot be operated or selected, but may be generated for a selectable object (the embodiment described later). See). For example, in the case of the pull-down menu described above, reaction force may be generated when passing through a selectable entry.

The pointer and the object for effecting the reaction force control may be arbitrarily set at the level of the OS and application software.

<Second Embodiment> In the above, an example in which the operation / input information is fed back by reaction force control based on the position information of the mouse (or of the pointer operated by the mouse) has been shown. It is also conceivable to add a control according to the pointer operated thereby) and a mouse movement speed.

FIG. 6 shows the structure of the computer system of the second embodiment. The difference from the first embodiment of FIG. 1 is that the mouse moving speed detecting section 2 is provided in the mouse driver 112. is there.

The mouse moving speed detecting unit 2 can detect the moving speed of the mouse by counting the position information from the mouse, usually a pulse signal within a predetermined time, and this signal can always be used in the host computer. Is done.

In the first embodiment, the user operates the mouse to move the pointer to generate a reaction force according to the position information during the movement, and all the reaction force control is set. Since a reaction force is constantly generated near the object, the user's feeling of use may be significantly reduced.
The second embodiment solves this problem.

A characteristic of the second embodiment is that the user pays attention to the operation of the mouse when it is operated.

Usually, an icon 34 is displayed by using a pointer 35 on a graphical user interface as shown in FIG.
When selecting b, it is known that the user moves the pointer quickly when the pointer is far from the icon and slows down the speed when the pointer approaches.

In the second embodiment, it is judged from the characteristics that the user changes the moving speed of the pointer as described above, whether or not the object is the object intended by the user.

FIG. 7 is a flow chart showing the state of mouse input / output control in the host computer 110 of the second embodiment. This control is performed by the CPU of the host computer 110
Is executed by

Step S71 is a speed judging means for judging whether or not the moving speed of the mouse is less than or equal to a predetermined value by comparing the values obtained from the mouse moving speed detecting unit 2. If the speed is less than or equal to the predetermined value, it is true (true). ), If it is greater than or equal to the predetermined value, it is determined to be false.

Step S72 is a mouse position judging means for judging the relationship between the position of the pointer and the position of an object such as an icon or a window. If the position of the pointer is within or near the object, it is true that it is a certain predetermined distance away. If false.

Step S73 is a reaction force control means for actually generating a reaction force in the reaction force control section 1 of the mouse 114.

As is apparent from FIG. 7, in the second embodiment, the result of the mouse speed judging means 71 is below the predetermined speed,
In addition, when the position of the pointer is within or near the object, the reaction force control means operates, and the reaction force control unit 1 generates a reaction force.

That is, even if the position of the pointer is inside or near the object, if the moving speed at that time exceeds the predetermined speed, it is judged that the user does not intend to operate the object. In that case, the reaction force control is not performed.

In the first embodiment, based on only the position information,
Since the reaction force control is performed, the reaction force is always generated in the vicinity of all the objects to which the reaction force control is set, which impairs the user's feeling of use. You can avoid the problem.

The processing procedure of the mouse speed determination means and the mouse position determination means of FIG. 7 may be reversed.

<Third Embodiment> This embodiment is different from the second embodiment in that when a mouse button is operated in the vicinity of the object, the object (nearby) position is passed and a reaction force is generated. This is a process in which the position information in which the operation has occurred is corrected by using the time information of.

Specifically, when the mouse button is pressed after starting the reaction force control in the second embodiment, the time from the start of the reaction force control in the vicinity of the object to the time when the mouse button is pressed is measured. If it is within a predetermined time, it is interpreted that the user has an intention to select the object, the position of the pointer is replaced with the position where the reaction force control is started, and the processing of the host computer is performed.

FIG. 8 is a flow chart showing the characteristics of the third embodiment. Steps S71 to S73 are the same as those in the second embodiment, but if the position of the pointer is in or near the object in step S72, the step is executed. In S74, the position (that is, the position where the reaction force control is started in step S73) is stored.

That is, step S74 is a coordinate position storage means for temporarily storing the position of the pointer when the reaction force control is started, and is normally stored in the memory mounted on the host computer 110.

Step S75 is a button determining means for determining whether or not the mouse button has been pressed. If the button is pressed, it is determined to be true, and if not, it is determined to be false.

Step S76 is a time judgment means, and compares the time from the reaction force control until step S75 as the button judgment means becomes true with a preset predetermined time, and if it is less than a predetermined value. True, if it is equal to or greater than a predetermined value, it is determined to be false.

Step S77 is a coordinate processing means for replacing (correcting) the coordinate position of the mouse currently in use with the coordinate value stored in step S74.

FIG. 9 shows an example of operation in this embodiment, and here shows an example of window operation.

90 is a window which is an object, 9
Reference numeral 1 denotes a size variable box for changing the size, shape, etc. of the window 90. Normally, the user can arbitrarily change the size and shape of the window by clicking and dragging with a mouse. In general, the variable size box is located at the lower right as described above, and when the drag operation is performed, the position of the upper left corner of the window is fixed, and the size or shape of the window 90 is the coordinate window of the final variable size box. It is determined to be a rectangle having a diagonal line defined by the coordinates of the upper left corner (the shape of the size-variable box itself, the position on the window, etc. are not limited to this, and various methods are used).

Normally, the window is designed and designed so that the inside display portion is enlarged in order to increase the contents to be displayed, and the variable size box of this window is no exception, and is designed to have a small shape as much as possible. It is usually done. However, it is more convenient to operate the mouse with a large size variable box, which is inconsistent with the above, and the larger the display area in the window, the more difficult it becomes to operate the size variable box.

Such a contradiction can be solved by this embodiment.

As is apparent from the control of FIG. 8, when changing the size of the window of FIG. 9, the pointer 35 passes through the variable size box 91 or passes in the vicinity thereof (step S72), so that the reaction force from the mouse. Is generated (step S73), the user performs an operation such as clicking or dragging a button in response to the reaction force feedback (feeling) (detected in step S76), and the reaction force is generated at the time of the operation. If the elapsed time of (i.e., passing the variable size box or passing in the vicinity) is within a predetermined time, the coordinate at the time of button operation is replaced with the coordinate information stored in step S74 (step S74).
77).

Therefore, if the reaction (button operation) of the user is obtained within a predetermined time according to the reaction force due to the reaction force generation, it is determined that the target object is operated even if the reaction is outside the target object. Since the coordinates of the operation position are corrected in the same manner as when the target object is operated by scanning, the same effect as when the target object is operated can be obtained.

Therefore, in the case of a variable size box,
It has the same effect as placing the pointer inside and dragging and clicking, eliminating the need for the user to accurately position the pointer in the variable size box.
The user interface device is convenient for the user.

That is, according to this embodiment, even if the user does not completely position the pointer on the object, once,
Passing on or near it, a reaction force is generated on the user, and the response and feel of the reaction force can be achieved by pressing the mouse button, which eliminates the need to concentrate attention on the screen display as in the past. Can reduce the burden on the eyes of the user when operating the graphical user interface.

In the present embodiment, the speed judgment in step S71 is not an essential requirement for correction of the coordinate position, and steps S74 to S77 are executed even if only position detection is performed as in the first embodiment. The treatments can be combined.

<Fourth Embodiment> In the above-described first to third embodiments, the reaction force control unit causes the mouse to generate a frictional force with respect to the detection ball of the mouse to apply a braking force to the progress of the mouse, and to the user. We are feeding back information from our host computer.

However, the means for feeding back information to the user is not limited to the above-mentioned reaction force control section. The present embodiment exemplifies a configuration in which a vibrating means for mechanically vibrating the mouse is provided and information is fed back to the user.

FIG. 10 is a partially cutaway side view of the mouse of this embodiment. Here, only the differences from FIG. 2 of the first embodiment will be described.

In FIG. 10, reference numeral 201 is a DC motor whose rotation can be controlled by passing an electric current, and 202 is an eccentric weight which is intentionally eccentrically attached to the rotating shaft of the DC motor 201. By controlling the rotation of the motor 201, the entire mouse can be vibrated. Reference numeral 200 denotes a vibration control unit equipped with a DC motor and an eccentric weight.

The reaction force control unit 1 of the first to third embodiments already described.
By replacing the vibration control unit 200, the same result can be obtained.

As the vibrating means used in the vibrating control section 200, in addition to the method of using a DC motor and an eccentric weight, 1) a weight is vibrated by a voice coil.

2) Mechanically hit the inside of the mouse.

3) The reaction force control of Examples 1 to 3 is frequently repeated.

4) Since a mouse is usually used together with a mouse pad (which is laid under the mouse), a vibration source is provided on the mouse pad.

The configuration described above is conceivable, and any means may be adopted as long as it can feed back the vibration to the user's hand.

Further, by using these vibrating means, when using on the graphical user interface shown in FIG. 3, 1) When the user drags an object such as an icon or a window, it is vibrated.

2) By providing the vibrating means with means for varying the vibrating intensity, the vibrating intensity is changed according to the characteristics of the object, for example, the type and capacity of the file associated with the object. Is possible. It goes without saying that the feedback by reaction force in the first to third embodiments can be completely replaced with the feedback by vibration in this embodiment.

<Fifth Embodiment> In each of the conventional examples described above, the configuration in which the mouse is used as the pointing device has been described. Needless to say, a similar configuration can be implemented in a pointing device other than a mouse.

FIG. 11 shows a laptop (notebook or sub-notebook) type computer that has become widespread in recent years. In such a computer 300, a so-called trackball is used for inputting coordinates by operating a spherical ball. There is one which provides 301.

In FIG. 11, reference numeral 300 is a computer of this embodiment, and 301 is a trackball, which is a feedback means (not shown) for feeding back information to the user by mechanical action such as coordinate detection means and control means or vibration means. have. It can be considered that the structure of the trackball is such that the detection ball of the mouse is directly operated by hand, and the feedback means for the trackball may have almost the same structure as the above-mentioned mouse feedback means. .

Reference numeral 302 is a liquid crystal display for displaying the graphical user interface shown in FIG. 3, and reference numeral 303 is a keyboard for inputting a character code.

For the trackball 301, a reaction force means similar to the reaction force control section 1 described in the first embodiment or a feedback means such as a vibration means similar to the vibration control section 3 described in the fourth embodiment is used. If the above-mentioned control is performed, the same effect as the effect described so far can be obtained.

When the vibrating means is used in the present embodiment, the ball itself of the trackball may be vibrated, the trackball may be touched by the operation of the computer, or the keyboard may be vibrated.

In addition, tablets, AccuPoint,
If it is a pointing device such as a joystick that is operated by the user's hand, it is possible to provide a means for giving feedback to the user's body by a mechanical action as described above, and it is possible to achieve the same effect as the above-mentioned embodiment.

Up to now, the description has been made on the user interface by the human hand, but the same device configuration is possible by using a part other than the hand, such as a foot.

[0097]

As is apparent from the above, according to the present invention, when the user operates the pointing device,
An operating environment that uses a feedback device that feeds back information to the user by mechanical action such as reaction force, vibration, and impact on a pointing device, and a graphical user interface that arranges objects such as icons, windows, and pop-up menus on the display. In, when the pointer operated by the pointing device passes or approaches the operation target object or its vicinity, the information about the operation of the pointing device is fed back to the user through the feedback means, and the feedback is responded to. Since the control means for processing the information about the operation of the pointing device of the user is provided, the information about the operation of the pointing device is provided to the feedback device. By providing a control means for feeding back information to the user via the means and processing information on the operation of the pointing device by the user in response to the feedback, the user interface via means other than the visual display is Since it is not necessary for the user to focus his attention on the display and accurately manage the position of the pointer as in the conventional case, a user interface with excellent operability can be realized.

[Brief description of drawings]

FIG. 1 is a block diagram of a computer system that employs a first embodiment of the present invention.

FIG. 2 is a partially cutaway side view of a mouse used in the device of FIG.

FIG. 3 is an explanatory diagram showing a graphical user interface to which the configurations of FIGS. 1 and 2 are applied.

FIG. 4 is an explanatory diagram showing control when operating a small icon in the first embodiment.

FIG. 5 is an explanatory diagram showing control when a pull-down menu is operated in the first embodiment.

FIG. 6 is a block diagram of a computer system adopting a second embodiment of the present invention.

FIG. 7 is a flowchart showing the characteristics of the second embodiment and the processing thereof.

FIG. 8 is a flowchart showing the features of the third embodiment and its processing.

FIG. 9 is an explanatory diagram when the third embodiment is applied to a window object.

FIG. 10 is a partially cutaway side view of the mouse of the fourth embodiment.

FIG. 11 is an explanatory diagram when the trackball of the fifth embodiment is used.

FIG. 12 is an external view of a conventional mouse.

FIG. 13 is a side view of the mouse of FIG.

FIG. 14 is a bottom view of the mouse of FIG.

FIG. 15 is a block diagram of a conventional computer system using a mouse.

FIG. 16 is an explanatory diagram showing a configuration of a conventional mouse movement amount detection unit.

[Explanation of symbols]

 1 Reaction Force Control Section 2 Mouse Movement Speed Detection Section 10 Friction Pad 11 Arm 12 Rotation Center 13 Magnet 14 Electromagnetic Coil 30 Outer Frame 31 Title Bar 32 Menu Bar 33 Window 34a-34e Icon 35 Pointer 60 Points 61 Line 62 Perpendicular 71 Mouse Movement Speed determination means 72 Mouse position determination means 73 Reaction force control means 74 Coordinate position storage means 75 Button determination means 76 Time determination means 77 Coordinate processing means 90 Window 91 Variable size box 100 Mouse housing 101 Mouse button 102 Mouse button 103 Lead wire 104 Detection Ball 105 Sliding member 106 Sliding member 110 Host computer 111 Operation system 112 Mouse driver 113 Mouse interface 114 Mouse 115 Movement amount detection unit 116 Button detection unit 120 Detection roller 121 Detection roller 122 Detection sensor 123 Detection sensor 124 Pressure sensor 200 Vibration control unit 201 DC motor 202 Eccentric weight 300 Computer 301 Trackball 302 Liquid crystal display 303 Keyboard

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inoue, Banyuki Isonuma 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. (72) Kouji Kawamura 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (14)

[Claims] 1. By moving on a predetermined plane,
In a user interface device that uses a pointing device to input information about the amount of movement, when the pointing device is operated by the user, the information is fed back to the user by mechanical action such as reaction force, vibration, or impact on the pointing device. In an operating environment that uses a feedback means and a graphical user interface that arranges objects such as icons, windows, and popup menus on the display,
When the pointer operated by the pointing device passes or approaches the object or the vicinity thereof, the information on the operation of the pointing device is fed back to the user through the feedback means, and the pointing of the user according to the feedback. A user interface device comprising a control means for processing information related to device operation. 2. The user according to claim 1, wherein an electromechanical conversion element is used as a part of the configuration of a means for generating a reaction force or generating a vibration as a feedback means by a mechanical action. Interface device. 3. Information feedback by the mechanical action via the feedback means when a pointer moved by operating a pointing device passes or approaches an object such as the icon, window, or pop-up menu, or in the vicinity thereof. The user interface device according to claim 1, wherein: 4. The information feedback by the mechanical action is performed via the feedback means when the target object passes or approaches the vicinity only when the size of the target object is smaller than or equal to a predetermined size. User interface device. 5. In the environment where the object is a menu bar and the menu bar is operated by a pull-down selected by dragging a pointing device and releasing a button, selectable entries and non-selectable entries in the menu bar are performed. If the entries are mixed,
When the pointer moved by the operation of the pointing device passes or approaches any one of the selectable entry and the non-selectable entry or the vicinity thereof, the information feedback by the mechanical action is performed via the feedback means. The user interface device according to claim 1, wherein: 6. A means for detecting the approaching speed of a pointer operated by a pointing device to the object is provided, and when the object is passed or approaches the object or its vicinity by using the detecting means, the object is detected by the user. 4. The method according to claim 3, wherein it is determined whether or not the object is a target object, and based on the determination result, whether or not to perform information feedback by the mechanical action to the user via the feedback means. User interface device. 7. A method for determining whether or not an object is an object intended by a user by the approaching speed of a pointer of a pointing device, and performing information feedback by the mechanical action via the feedback means according to the determination result. , A means for measuring the time from the feedback of the information by the mechanical action until the button of the pointing device used by the pointing device to select an object is pressed, and the time of the measuring means is a predetermined time. The user interface device according to claim 6, wherein if it is within the range, control is performed to consider that the object is selected. 8. A means for temporarily storing a coordinate position at the time when the mechanical operation of the pointer operated by the pointing device is started, that is, when the pointer passes or approaches the object or its vicinity, When the button of the pointing device is pressed within the predetermined time measured by the measuring means, the coordinates of the pointer obtained when the button of the pointing device is pressed by the coordinate information temporarily stored in the storage means. The user interface device according to claim 7, wherein the copying control is performed by replacing information. 9. When the user operates an object by clicking or dragging a pointing device in order to change the size of the window in the object or the like in the object, the feedback means is used to operate the object. Information feedback control by a mechanical action is performed, Claims 1-8.
The user interface device according to 1. 10. The mechanical action vibrates a pointing device, and an eccentric weight is provided on a motor rotation shaft for a rotation controllable motor, or a weight is vibrated by a voice coil, or a piezoelectric element or a solenoid. 10. The user interface device according to claim 1, wherein the vibrating action is generated by hitting the inside of the housing of the pointing device with an active element such as. 11. The user interface device according to claim 10, wherein, when an object such as an icon or a window is dragged, information is fed back by the mechanical action via the feedback unit. 12. The user interface device according to claim 10, wherein the strength of action of the feedback means is changed according to the characteristics of an object such as an icon or a window, for example, the size or type of a file. . 13. The mouse according to claim 1, wherein the pointing device is a mouse device having means for detecting movement of the device and an input button.
2. The user interface device according to any one of 2 above. 14. The pointing device is a trackball device which has a button for inputting coordinate information to a computer by a user rotating a spherical ball by hand and having an input button.
~ The user interface device according to claim 12.