JPH03186979A - Posture of hand recognition system using neuro-computer - Google Patents

Abstract

PURPOSE:To give instructions to a computer by humanlike expression at the time of the recognition of the posture and the state of a hand by using a neuro- computer. CONSTITUTION:A posture of hand measuring part 1 measures the posture or the state of the hand, and a posture storage part 2 stores the posture and the state of this hand as generalizing them. Further, a posture recognizing part 3 recognizes the posture and the state of the hand according to data from the posture of hand measuring part 1. Then, it is constituted so that it recognizes the various kinds of the postures and the states of the hand to be inputted newly according to the combined weight of a neural network stored by the posture storage part 2 on the basis of the various kinds of the postures and the states of the hand inputted beforehand. Thus, the recognition system of a wide allowable range is realized by generalizing the posture and the state so that the various kinds of the postures and the states of the hand can be recognized, and simultaneously, correction and addition too can be executed very simply.

Description

[Detailed Description of the Invention] [Summary] Regarding a method for recognizing hand postures and conditions in human interfaces between humans and computers, a recognition system that generalizes hand postures and conditions and has a wide tolerance range is provided. The aim is to provide a hand posture recognition method that can easily perform corrections and additions to the hand posture and condition. It has a posture memory section using a neurocomputer that generalizes and stores the posture and state of the hand, and a posture recognition section that recognizes the posture and state from the data from the hand posture measurement section. Based on the various postures and conditions, the configuration is such that various postures and conditions of newly input hands are recognized by the connection weights of the neural network stored in the posture storage section.

[Industrial application field]

The present invention provides a human interface (man-machine interface) between humans and computers.
Concerning how humans give instructions to computers.

As computers have become more sophisticated and functional in recent years, a complete interface between humans and computers (referred to as a human interface or a man-machine interface) is becoming increasingly important.

Currently, typical interfaces such as Keypo F, mouse, tablet, etc. are very difficult for humans to operate, and humans are currently trying to get used to these interfaces. It is.

For example, when trying to input numbers into a computer,
Numbers must be entered from the keyboard.

It is more natural to enter this number by, for example, holding up three fingers and manually inputting 3.

Also, when the cursor does not move slightly, it is less troublesome to be able to point the mouse at a flat surface on the desk, such as raise the index finger up, middle finger up and down, etc.

This is necessary as a human-machine interface, and the technology for this is required.

In response to this demand, gloves called data gloves have been developed in recent years that can measure the posture and condition of the hands. Therefore,
Using this data glove, we measure the posture and condition of our hands,
By recognizing the current posture of a person, it will be possible to notify the computer of what the person is requesting.

With this in mind, we will be able to create human-man and man-machine interfaces that are extremely human-like, hassle-free, and easy to separate. Therefore, an effective hand posture recognition method using this data glove is needed.

[Prior art and problems to be solved by the invention] FIG. 3 is a diagram illustrating a conventional human-machine interface.

As shown in this figure, conventional human-man interfaces and man-machine interfaces for boat fishing have mainly been keyboards, mice, cubes, etc.

Therefore, humans had no choice but to give instructions to computers using only these devices. There was no way to manually input numbers, instruct movements, or teach the five positions to computers in a way that was easy to understand, and there was no device that was easy for the humans operating them to understand.

A major problem with this was that the method of measuring hand posture and condition was difficult. However, a device called a data glove has recently been developed to measure this, and there is a movement to make this a standard method.

For example, (1) for the 3-year-old data globe.
Data Glove Type 2 Operation Manual” Document Data, August 25, 1989.2.2 edition.

Published by VPLriF Kyusho, I”DATAGLOVE M
ODEL 20peration Manual'',
Document date: August 25
．． 1989. Version 2.2. Copyrig
ht: 1989 by VPL Re5earch
Inc.) (2) "Data Globe Program Creator's Guide", Document Data: June 1989, 1.3 edition, published by Sim Graphics Industrial Company, ("Data Globe" Pro
Grammers Toolkit, Programm
ers Gurde”, Document date:
June 1989. Version 1.3. Co
pyright: ]989 by SimGraph
ics Engineering Corporation
on) etc.

The methods described in these documents attempt to recognize angle data from a data glove that indicates many postures by using a program in each case.

Therefore, the system must be created, and if a request to identify two other postures occurs after the system is created, there is the hassle of having to create a new recognition system. For example, when you raise your little finger, you may want to register a new command to cancel the current human power.
We assumed that when the index and middle fingers were held up, it would indicate 2, but if we wanted to change the number when the thumb and index finger were held up, we needed to redesign the entire system.

Furthermore, if the information is created analytically using a program, it may be possible to recognize a specific person, but if the person changes, it may become unrecognizable.

That is, in the conventional technology, since a system for recognizing hand postures and conditions is constructed using a program, the system can only be applied to specific postures and conditions. Therefore, it has to be rebuilt to accommodate changes and additions to hand postures and conditions, and since the program is divided into cases for different postures and conditions,
There is a possibility that the posture state that has been passed through 49 times will be recognized differently, and in order to solve this problem, a large number of processes are required, such as rebuilding the program again.

In view of the above-mentioned conventional drawbacks, the present invention generalizes these postures and conditions so that various hand postures and conditions can be recognized, thereby realizing a recognition system with a wide tolerance range, and is also highly resistant to modifications and additions. The purpose of this invention is to provide a hand posture recognition method that can be easily performed.

[Means for Solving the Problems] FIG. 1 is a diagram explaining the present invention in detail, in which (a) shows an example of a system configuration, and (b) shows an example of the bottom of a neural network learning device and a recognition device. , (c) shows the input/output characteristics of neural phone 1 to work, and (d) shows an example of the structure of neural phone 1 to work.

The above problems are solved by a hand posture recognition method using a neurocombinator constructed as follows.

In a human interface between a human and a computer, there are a hand posture measurement section 1 that measures the posture and condition of the hand, and a posture storage section 2 using a neurocomputer that generalizes and stores the posture and condition of the hand. and a posture recognition section 3 that recognizes the posture and state of the hand from the data from the hand posture measurement section 1.

Based on the various postures and states of the hand that have been input in advance, the various postures and conditions of the hand that are newly input are determined by the connection weights of the neural network stored in the posture storage section 2.
Configure to be state aware.

[Effect]

That is, according to the present invention, a "hand posture recognition device using a neurocomputer" includes a "hand posture measurement device" and a "neurocomputer learning and recognition device," as shown in FIG. 1(a). ”. The ``neurocomputer learning and recognition device'' consists of a ``neural computer learning device'', a ``connection weight storage section'', and a ``recognition device'', and operates as shown below.

■Hand posture measuring device Hand posture measuring device uses optical fiber to measure the posture and condition of the hand using the data glove as the center. (Refer to the above-mentioned documents (1) and (2)) Specifically, the angles of the first and second joints of the fingers can be measured, and ten pieces of data can be obtained for five fingers. The angle between each joint is 90 degrees in the "goo" state and 0 degrees in the "par" state.

First, several poses of postures that are desired to be recognized during learning are generated and notified to the neurocomputer learning 5 recognition device described below. All posture.

After learning the state, if data of a certain posture is notified to the recognition device, recognition of that posture can be obtained as a result.

Addition of a new posture or state or change of recognition can be easily achieved by learning the posture or state again.

Next, three devices in the ``neurocomputer learning and recognition device'' will be explained. (See FIG. 1(b)) Neural network learning device 0 The neural network learning device consists of a learning pattern holding section, a neural network execution section, and a two-weight updating section. The learning pattern holding section holds an input pattern and a desired output pattern (teacher signal) corresponding thereto.

A neural network that applies the Hank Bropage Soyon method is a multilayer network.

Each layer is composed of many units, and the units in each layer are connected to each other with a certain weight.

The weights of the network can be learned by providing a set of human cover turns and desired output turns (teacher signals) to this network.

Learning proceeds as follows. Give an input to the neural network and get the output. If the output is incorrect, the correct (desired) output value is taught to the neural network.

Then, the neural network adjusts its internal structure (strength of connection-weight) so that the difference between the correct output and the actual output value is reduced. By repeating this many times, the neural network automatically learns the weights that satisfy the input-output relationship. This learning algorithm is called the bank propagation method.

By using the neural network trained in this way, it will return the correct output as taught for input cover patterns that have been learned, but it can also return output cover patterns that are interpolated from the learned input and output cover patterns for input cover patterns that have not been learned. . This is a major feature of neural networks.

A neural network learning device (see FIG. 1(b)) that performs this learning will be described below.

As described above, the neural network learning device includes a learning pattern holding section, a neural network execution section, and a first layer updating section.

(1) Learning pattern holding section input cover turn and its desired output cover turn (
teacher signal).

(2) Neural network execution unit It has a multi-layer network structure. Each layer is composed of many units, and a connection weight W is defined between each unit.

Each unit calculates the network output value as shown below.

When a unit receives input from multiple units,
The sum total plus the threshold (aO) of the unit becomes the input value net (see Figure 1(c)), that is, the unisol
−U i's input neJ is net H−ΣW;ioj
+(El.

Wl,...: Weight of connection from unit UJ to unit U3 ○j: Output of unit 1-Uj Ol: Threshold of unit U The output value of the unit is the sum of this input net t.By applying the activation function, Calculated. When the activation function is a differentiable nonlinear function, for example, a sigmoid function, the output value O8 of the unin 1-U becomes 3.

The network used in the bank propagation method is a multi-layer network for boat fishing, but here we will discuss the case of a three-layer network as shown in FIG. 1(d), which is commonly used.

The three layers are called the input layer, hidden layer, and output layer, and each layer is composed of many units. Each unit of the hidden layer is connected to all units of the input layer. Each unit in the output layer is connected to all units in the input and hidden layers. There is no connection within each layer.

Each unit in the human layer is given input data to the network.

Therefore, the output value h4 of each unit U in the hidden layer is netj −ΣwJkd, + eJ The weight of the connection between the human unit and the j-th hidden unit ■,; the dark value of the j-th hidden unit 1~. Output value of each unit in the output layer. , from the above formula, 1 + exp (-net + )h,
+: output value of the j-th hidden unit oHsi-th output value of the output unit Wij' weight of the connection between the j-th hidden unit and the i-th output unit ○, : is the threshold value of the ]-th output unit.

(3) Weight update unit This is a part that changes the weight of the network so that the output of the network becomes a desired output.

Mean squared error E between the actual output value (op,) and the desired output value (t+, i) when a certain pattern p is given.

Take.

■ BP-(t pi Opt)” In order to learn a certain pattern p mentioned above, all the weights in the network are changed so as not to reduce this error.

-Learning rules for the output layer (learning rules to eliminate the above error) 1) Change in weight between unit UJ in the hidden layer and unit U in the output layer Δw8. (n) is △wij (n)
= yyΣδ, (h pj + tr△WfJ(nl
) 11) The weight change Δw,k(n) between the unit U in the input layer and the unit U in the output layer is ΔWih (n)
= ηΣδpi d ph + rx△W, k (nl) where n: number of learning α: momentum (smoothness of the sigmoid function above) δpt-(jp;-Op4> (Op, (1~op,
)] Learning rule for hidden layer The change in weight between input layer unit Uk and hidden layer unit Uj is Δw = w (n) = ηΣδpj h pj
+αΔWjk(nl) δ,, t=hpj (1−h,j)Σδ1Wij.

In order not to destroy the above mean square error E, the above weight changes ΔWij, ΔW86. Learning is performed by calculating Δwjk and providing it to the neural network.

The formula giving the output value of each unit in each input layer, hidden layer, and output layer, or the output layer, l! ! For details on the process of deriving the learning rule formula in this layer, see, for example, rPDP
Model “Cognitive Science and the Exploration of Neuronal Networks”, D, E
, Lamelber 1-, J.L., McClelland, PDP Research Group, supervised translation by Toshi Seri, February 27, 1989, Sangyo Tosho Co., Ltd., first edition, so only the results will be shown here. Only.

(2) Connection Weight Storage Unit The weight storage unit stores the weights between each unit of the neural network at any learning stage.

■ Recognition device The recognition device loads the weights of the neural network stored in the weight storage unit, and when an input pattern different from that during the learning is given, calculates the output of the neural network determined by the weight of each unit. . When the network trained as described above is used, for input cover patterns that have been learned, the correct output as taught above will be returned, but for input cover patterns that have not been learned, output patterns that are based on the learned turn cover patterns will also be returned. It functions to return .

As explained above, in this method, by using a neural network to learn *number patterns of hand postures and states, the trained neural network 8 - generalizes the postures and states. The weights are stored in the weight storage. Therefore, special@,
It becomes possible to store postures and states that are the same (slightly bent, stretched, etc.) as weights in one network. It is also possible to store postures and states with different characteristics as weights for different networks.

When recognizing the posture and state of the hand, first, the learned network weights in the weight storage section are loaded into the Noodle network. By using the weights of the net learned in this way, by specifying the joint data of the input hand posture and condition, it is possible to recognize what the posture and condition indicate.

Furthermore, even if you want to newly register a posture or state, or if you want to change a posture or state that has already been learned, it is possible to easily recognize the posture or state by relearning the posture or state.

〔Example] Embodiments of the present invention will be described in detail below with reference to the drawings.

The above-mentioned FIG. 1 is a diagram for explaining the present invention in detail, and FIG.
The figure shows an embodiment of the present invention, in which a neural network learning device 20 is used to learn the posture and condition of the hand based on the joint data output from the data glove, and the learning is completed. The connection weights between the units constituting the neural network at the time are stored in the connection weight storage unit, and when the attitude and state of the nucleon are to be recognized, the connection weights are stored in the connection weight storage unit 2. A means necessary to implement the present invention is to load learned connection weights into a neural network and recognize new joint data output from the data globe. Note that the same reference numerals refer to the same objects throughout the plan.

Hereinafter, with reference to FIG. 1 and FIG. 2, a hand posture recognition method using the neurocombinator of the present invention will be explained.

First, in the ``Second System'', it is necessary to make the neural network learn the seven states of hand posture. for example,
To represent numbers, learn the numbers with your fingers up. When only the index finger is used, “]J, and when the middle finger is raised, “
The hand posture measuring device 1 collects data that has been gradually sensitized for all patterns, such as ``2'' when the ring finger is up, ``3'' when the little finger is up, ``4'' when the little finger is up, and ``5'' when all the fingers are open.
For example, 10 patterns are measured. The reason why 10 patterns are prepared is to learn as many patterns as possible so that G-como recognition can be performed normally when the person changes.

FIG. 2 shows a learning method for learning the hand posture, a' position. When the neural network is trained, the learning pattern taught to the neural network is, for example, the second joint angle (84, 20, 23, 39, 29)
Joints (14, 79, 82, 2L 43) 1st joint When the parent middle ring pinky finger finger finger, the posture at this time of the crab is the number "2".

The pattern you want to recognize (for example, holding up your index and middle fingers)
In order to input 2'' into the neural network, the neural network learning device 20 shown in FIG.
For example, '000010'') and use it as a teacher signal (same as the recognition pattern).

Then, this bit pattern is used as the input X (teacher signal) of the neural network. At this time, the joint angle indicating 2'' outputted from the hand posture measuring device 1 is set as the input Y (learning pattern) of the neural network.Thereby, the neural network will be close to the teacher pattern X when the input Y is Outputs the connection weight of the network that outputs the recognition pattern (binary number).

If a plurality of postures and states are to be learned, a plurality of learning patterns Y as shown in FIG. 2 are to be learned. The weights of the neural phoneworks that have been trained are stored in the connection weight storage unit 21, and when it is desired to recognize the posture using the stored neural phoneworks, the connection weights are stored in the neural phoneworks. component work.

2 After learning, by inputting the hand posture and state from the hand posture measuring device (data glove) 1, the corresponding recognition results (“1” to “5”) can be obtained.

Furthermore, in addition, when it is decided to make a circle with the thumb and forefinger to indicate "OK", the data of that state is re-learned, for example, 10 patterns.

After learning this pattern, the connection weights of the neural network are held in the connection weight storage section, so that this state can also be recognized normally during recognition.

〔Effect of the invention〕

As explained above, in order to recognize the posture and condition of the hand,
By using a neurocomputer, it becomes possible to generalize and memorize hand postures and conditions, and therefore it becomes possible to recognize appropriate postures and conditions under various environments. As a result, a system can be constructed in which instructions can be given to a computer using more human-like expressions in a human interface, and the recognition can be easily changed or added.

Therefore, the number of man-hours required for creating an interface system can be reduced, and a human interface that is more human-like can be provided.

[Brief explanation of drawings]

FIG. 1 provides a detailed explanation of the present invention, FIG. 2 shows an embodiment of the present invention, and FIG. 3 is a diagram illustrating a conventional human interface and man-machine interface. It is. In the drawings, 1 is a hand posture measuring device, or a data graph, or a posture measuring section 2 is a neurocomputer, and a recognition storage section 20 is a neural network learning device. 200 is a neural network execution unit 201; learning pattern holding unit 202 is a weight updating unit. The connection weight storage unit 3 21 represents a recognition unit or a recognition device, respectively. 5

Claims (1)

[Claims] In a human interface between a human and a computer, there is provided a hand posture measurement unit (1) for measuring the posture and condition of the hand, and a generalization and storage of the posture and condition of the hand. It has a posture memory section (2) using a neurocomputer, and a posture recognition section (3) that recognizes the posture and state of the hand from data from the hand posture measurement section (1), and can handle various hand inputs in advance. A neurocomputer is used which is characterized in that it recognizes various newly input postures and conditions of the hand based on the connection weights of the neural network stored in the posture storage section (2), based on the postures and conditions of the hand. Hand posture recognition method.