JPH04340669A - Video generation method for multi-jointed structures - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for generating a moving image of a three-dimensional object, and more particularly to a method suitable for generating a moving image using key frames of a multi-jointed structure.

0002

2. Description of the Related Art In the field of computer graphics, when moving a three-dimensional object, dynamics may be considered in order to obtain realistic movement. In particular, dynamic calculations are often useful when generating the motion of a multi-jointed structure that has many joints and moves according to changes in the joint angles.

When performing mechanical calculations for multi-jointed structures, joint angles, joint angular velocities, and joint angular accelerations are used, as discussed in Measurement and Control, 25, 1 (1986), pages 23 to 29. Inverse dynamics calculations are used to calculate joint torques from joint angles, and dynamic calculations are used to calculate joint angular accelerations from joint angles, joint angular velocities, and joint torques.

0004

[Problems to be Solved by the Invention] In order to obtain realistic motion, not only inverse dynamic calculations but also dynamic calculations are necessary. However, dynamic calculations required a lot of calculations. Furthermore, it was difficult to impose a condition on the movement obtained by dynamic calculations, such that it takes a certain posture at a certain time. Therefore, although moving images obtained by dynamic calculations have more realistic movements than moving images obtained using key frames, it is difficult to control detailed movements.

[0005] An object of the present invention is to provide a moving image generation method that can easily generate realistic movements that take dynamics into account from key frames in moving image generation processing for multi-jointed structures.

[0006]

[Means for Solving the Problems] In order to achieve the above object, an initial value is given to the time required between key frames, and a cubic function is used to interpolate the joint angle between key frames. and,
The joint torque between key frames is determined by inverse dynamics calculation. If the joint torque exceeds the allowable value, the joint angular acceleration is corrected by multiplying it by a real number. If a movement in which the joint torque does not exceed the allowable value is not obtained within the given required time, a correction is made to change the required time, and the process returns to the process of interpolating joint angles between key frames.

[0007]

[Operation] By interpolating joint angles between key frames using a cubic function, smooth movements between key frames can be used as initial data. Inverse dynamics calculations can detect parts of the initial data where joint torque is unnaturally large. By multiplying the joint angular acceleration by a real number, the joint angular acceleration can be corrected with a small amount of calculation without performing dynamic calculations. By changing the time required between key frames, it is possible to find the time required for the movement between key frames to become natural. As described above, realistic movements that take dynamics into consideration can be easily generated from key frames.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the concept of the process of the present invention, and FIG. 2 shows an overview of the process. A key frame (101) consisting of data of each joint angle (103) at a certain time of the multi-joint structure
), a key frame (102) consisting of data of each joint angle (104) representing the next posture, and a range (108) of permissible joint torque of each joint between them. Under the condition that the joint torque (107) does not exceed the allowable value range (108) by the multi-joint structure moving image generation program (204) including inverse dynamic calculation,
Time required between keyframes 101 and 102 (106)
and a joint angle (105) that interpolates between key frames 101 and 102. By program 204, 10
A set of 105 values at a certain time is obtained from the key frame data (201) of the multi-joint structure, which is a set of 1 and 102, and the joint torque tolerance value (202) of the multi-joint structure, which is a set of 108. One frame of data (203) of the multi-jointed structure is sequentially obtained.

[0009] In the present invention, a moving image refers to a set of consecutive frames at sufficiently short constant time intervals. Among these frames, a frame in which the user directly gives the posture of the multi-joint structure in that frame is called a key frame. Data of frames between key frames is obtained by interpolating the posture of the articulated structure in the key frames.

FIG. 3 shows a system configuration when the process shown in FIG. 2 is applied to animation production of a human body model. Using this system, it is possible to efficiently create animations in which human models move realistically. The user uses an input device (306) such as a mouse or a keyboard to execute the multi-joint structure moving image generation program (204), and generates the key frame data (301) of the human body model corresponding to the data 201 and the data. From the joint torque tolerance value (302) of the human body model corresponding to data 202, one frame of data (3) of the human body model corresponding to data 203 is calculated.
03) are obtained sequentially. A set of data 303 is data to be interpolated between key frames. The initial value of the time required between key frames is given in the program 204 or input by the user through the input device 306. Background data (307) and data 303 are converted into a three-dimensional object display program (308)
is input and displayed on the display (309), and the user performs the operation while looking at this. The contents of the data 303 are
The animation data input/output program (304) stores the animation data (305) of the human body model. After the execution of program 204 is completed, data 3
By sequentially calling the contents of 05 using the program 304, the produced animation can be played back.

The details of the processing of the present invention will be explained below.

[1] Definition of multi-joint structure The multi-joint structure used in this embodiment is shown in FIG. Let Lc be a link connecting the joint Jc and the joint Je located on the distal end side. Further, the portion at the tip of the joint Jend at the tip is assumed to be a link Lend. Joint Jc has a coordinate system Fc with Jc as the origin. The relative position of Fc with respect to the coordinate system Fb of the joint Jb located on the base side is expressed by a 4×4 coordinate transformation matrix Mc. If the joint of the base is Jbase, then Mbase
is a coordinate transformation matrix representing the relative position of Fbase with respect to the world coordinate system Fworld. Using these coordinate transformation matrices, the position of Jc represented by Fworld is determined.

The degree of freedom of each joint of the multi-joint structure handled in this embodiment is 1. A two-degree-of-freedom joint is considered by replacing it with two joints and a link connecting them with length 0 and mass 0. Similarly, a joint with three degrees of freedom has three joints,
Consider replacing them with two links of length 0 and mass 0 connecting them. It also allows for branching from the base to the tip. That is, for a certain link Lc, there may be a plurality of links Le on the tip side. therefore,
The number of base links Lbase is one, but the number of tip links Lend is one or more.

[2] Inverse dynamics calculation When the joint angles, joint angular velocities, and joint angular accelerations of all the joints of a multi-joint structure are given at a certain time, the joint torques of all the joints can be calculated by inverse dynamics calculation. Desired. In this embodiment, a procedure for inverse dynamic calculation using the Newton-Euler method will be described.

The amount required for the calculation is defined as follows. Let θc be the joint angle of Jc. Let τc be the joint torque of Jc. Let mc be the mass of Lc. Let ωc be a 3×1 vector representing the rotational speed of Jc. Let zc be a 3×1 vector representing the rotation axis of Jc. Let vc be a 3×1 vector representing the translational speed of Jc. Let uc be a 3×1 vector representing the translational speed of the center of gravity of Lc. Let rc be a 3×1 vector from Jb to Jc. Let sc be a 3×1 vector from Jc to the center of gravity of Lc. Let fc be a 3×1 vector representing the force acting from Lc to Le. fend represents a force acting on Len from the outside. nc is 3× representing the moment acting from Lc to Le.
Let it be one vector. nend represents a moment acting on Lend from the outside. 3×3 for the rotation of Ac from Fc to Fworld
Let it be a transformation matrix. Ac corresponds to the rotational 3×3 submatrix of the product from Mbase to Mc. Let Ic be a 3×3 inertia tensor around the center of gravity of Lc, denoted by Fc. Let g be a 3×1 vector representing gravitational acceleration. Let E be the total number of Jends, that is, the total number of Lends. Let Kf be a 3×E matrix consisting of E column vectors representing forces. The i-th column vector of Kf represents the force acting on the i-th Len from the outside. Let Kn be 3 consisting of E column vectors representing moments.
xE matrix. The i-th column vector of Kn represents the moment acting on the i-th Len from the outside. Let N be the total number of joints, ie, the total number of links. Let Θ be an N×1 vector whose i-th component is θi. Let T be an N×1 vector whose i-th component is τi.

Link Lc has radius wc, height hc, Fc
When expressed as a uniform-density cylinder with the y-axis as the rotation axis, the component Iij at the i-th row and j-th column of the inertia tensor Ic of Lc is as follows.

I11=(mc・hc2+3mc・wc2)/12
...(Number 1)
I22=(mc・wc2)/2

...(Math 2) I33=(mc・hc2+3mc・w
c2)/12
...(Math 3) I12=I13=I21=I23=I
31=I32=0
...(Equation 4) The inverse dynamics calculation is performed according to the procedure shown below. In addition, in the following formula, ' represents the first-order differential with respect to time, `` represents the second-order differential with respect to time, and ^ represents transposition.

(1) ωbase, ωbase', vba
The initial value shown in Equation 5 is given to se'.

ωbase=0, ωbase'=0, vbase'
=-g...(Number 5
) According to Equation 5, an offset corresponding to the gravitational acceleration is given vertically upward to vbase'.

(2) Repeat calculations from Equation 6 to Equation 9 from the base to the tip.

ωc=ωb+zcθc'

...(Math. 6) ωc'=ωb'+zcθc"+ω
b×zcθc'
...(Math. 7) vc'=ωb'×rc+ωb×(
ωb×rc)+vb'
...(Math. 8) uc'=ωc'×sc+ωc×(ωc×
sc)+vc'...(Math. 9) (3) i-th fend and i-th nend
An initial value shown in Equation 10 is given to (1≦i≦E).

[0022] fend=Kf·ei^, nend=Kn·ei^
…(Number 10
) In Equation 10, ei is an E×1 unit vector such that the i-th component is 1 and the others are 0.

(4) From the tip to the base, the number 11
Repeat the calculations from to number 13.

fc=mcuc'+Σfe

...(Math. 11) nc=AcIcωc'+ωc×(Ac
Icωc) +sc×mcuc'+Σ((r
e×fe)+ne)…(Math. 1
2) τc=zc^・nc

...(Equation 13) T is determined by the above procedure. This calculation is expressed as shown in Equation 14 using the function Inv_dynamics.

T=Inv_dynamics(Θ, Θ', Θ'',
g, Kf, Kn) ... (Math. 14) [3]
If T obtained by the joint angular acceleration correction number 14 exceeds the allowable value of the joint torque of each joint, the joint angular acceleration Θ'' is corrected so that T falls within the allowable value range.This correction In order to efficiently perform
It is necessary to perform an operation to obtain ``.The relationship shown in Expression 15 holds between Θ'' in Expression 14 and T, so this operation can be performed by using this relationship.

HΘ”=T−B

...(Equation 15) H is called an inertia matrix, and B is called a bias vector. B is an N×1 vector and is obtained by equation 16.

B=Inv_dynamics(Θ, Θ', 0, g
, Kf, Kn) ... (Equation 16) H is an N×N matrix, and its i-th column vector Hi is expressed as Equation 1
7 (1≦i≦N).

Hi=Inv_dynamics(Θ, 0, ei,
0,0,0)...(Equation 17) In Equation 17, ei is 1 for the i-th component and 0 for the others.
is an N×1 unit vector such that . When the i-th component of B is bi, bi represents the joint torque of Ji when the influence of joint angular acceleration is ignored. Also, Hi is
It represents the joint torque of Ji when the joint angular acceleration of Ji is set to 1 and the other joint angular accelerations are set to 0, ignoring the effects of joint angular velocity, gravity, and external force.

By determining the bias vector B and the inertia matrix H, Θ'' can be determined from T. However, H
A lot of calculations are required to find . Therefore, in the present invention, we focus on the fact that H and T-B are proportional, and correct Θ'' by finding only B. Let the maximum absolute value of the joint torque of joint Ji be λi, and the i-th component is Let Λ be an N×1 vector such that λi.For all i (1≦i≦N
), the one closest to 1 is set as ko.

-λi-bi≦k(τi-bi)≦λi-bi
...(Math. 18)k
The calculation for obtaining o is performed using the function Get_coef as shown in Equation 1.
Expressed as 9.

ko=Get_coef(Θ, Θ', Θ", g, K
f, Kn, Λ) ... (Equation 19) Equation 19 is
This means that when Θ, Θ', and the target Θ'' are given, a joint angular acceleration ko times Θ'' can be achieved within the limit of Λ. Therefore, let koΘ" be the corrected joint angular acceleration. In some cases, ko cannot be obtained, and in this case, the function Get_coef returns φ for convenience. In other words, it is expressed as ko=φ. ko=φ In this case, the target Θ'' is changed and the calculation for finding ko is repeated. The target Θ'' is Θo'', and the joint angular acceleration correction procedure in the present invention is summarized as follows. (1) From Θ, Θ', Θo'' at a certain time, the formula 14
Find T by (2) Similarly, Θ, Θ', Θ
o'', find B using equation 16. (3) From T, B, Λ, find ko using equation 18. Steps (1) to (3) are the function Get_coef.
corresponds to (4) Θo” multiplied by ko is the corrected joint angular acceleration.

When ko=φ, Θo'' is changed and steps (1) to (3) are repeated. How to give Θo'' will be described in [5].

[4] Interpolation of joint angles between key frames In the present invention, data for interpolating key frame times and joint angles between key frames is provided, and [3]
Process. It is assumed that the times of three adjacent key frames are set to t0, t1, and t2, and the joint angles of Ji at each time are θ0i, θ1i, and θ2i (1≦i≦N). Using these values, the interval [
t0, t1] is determined. A function Thetai(t) representing Ji's joint angle at an arbitrary time t of [t0, t1] is determined. Thetai(
t) is a cubic function shown in Equation 20.

Thetai(t)=ci3・t3+ci2・t2
+ci1·t+ci0 (Equation 20) However, Thetai(t) is continuous up to the first derivative with the function of the adjacent interval at both ends of the interval. The four coefficients ci3, ci2, ci1, and ci0 in Equation 20 are determined using four of the five conditional expressions in Equation 21 to Equation 25.

Thetai(t0)=θ0i

...(Math. 21) Thetai'(t0)=θ0
i'
...(Math. 22)
t1)=θ1i
...(Number 23)
Thetai(t2)=θ2i

...(Math. 24) Thetai'(t1)=0

...(Equation 25) Which conditional expression to use is determined by Equation 26.

Di=(θ2i−θ1i)・(θ1i−θ0i)
…(Number 26
) However, if the key frame at time t1 is the last key frame, Di=0. When Di>0, use Equation 21, Equation 22, Equation 23, and Equation 24 to
Determine the coefficients of tai(t). When Di≦0, Equation 21, Equation 22, Equation 23, and Equation 25 are used. Di≦0
By using Equation 25 when , Thetai(t) has a local maximum value or a local minimum value at time t1. As a result, when the direction of the joint angular velocity is reversed before and after the key frame, the joint angle changes without exceeding the angle given by the key frame.

[5] Correcting the joint angle between key frames Once the function representing the joint angle in the interval [t0, t1] is determined, the joint torque in this interval is determined. At the time when the joint torque exceeds the range Λ, the joint angle acceleration is corrected, and the joint angle is corrected based on this. This process is performed at t
The process is performed sequentially from 0 to t1 at sufficiently short fixed time intervals Δt.

[0038] Current time is tc, next time is td=tc
+Δt. Let the joint angles at times tc and td be Θc and Θd, respectively, and the target value of the joint angular acceleration at time tc be Θo''. The joint angles are corrected by repeating the procedure shown below.

(1) First, assume that Θc and Θc' are given.

(2) Θd is defined by the function Thetai (1≦i
≦N).

(3) From Θc, Θc', and Θd, calculate Θo'' using Equations 27 and 28.

Θd'=(Θd-Θc)/Δt
...(Math. 27) Θo"=(Θd'-Θc')/Δt
…(
Equation 28) (4) From Θc, Θc', Θo'', find ko using Equation 29, and substitute this into Equation 30 to determine Θc''.

ko=Get_coef(Θc, Θc', Θo",
g, Kf, Kn,, Λ) …(Equation 29) Θc”=
koΘo”
…(Number 3
0) (5) Θd is obtained from Θc', Θc'' using Equation 31 and Equation 32.

Θd'=Θc'+Θc"・Δt
...(Math. 31) Θd=Θc+Θd'・Δt

...(Equation 32) When ko=1, Θd obtained in step (2) and Θd obtained in step (5) match, and when ko≠1, they do not match. Therefore, Θ only when ko≠1
d is modified. Note that if ko=φ during the above procedure, the process is interrupted at that point.

[6] In the process of correcting the joint angles in the correction interval [t0, t1] of the required time between key frames, ko
Assume that ko=1 does not hold and td=t1. At this time, joint angle data that matches the key frame at both ends of the section and in which the joint torque within the section does not exceed the range Λ is obtained. If such data is obtained, the process proceeds to the next section. However, if this is not the case, t1 and t2 are shifted by an integral multiple of Δt, and the processes [4] and [5] are performed again. t1
, t2 are initial values to1 and to2, respectively, and p is 0.
If the above integers are used, t1 and t2 when performing the processes [4] and [5] can be expressed by Equation 33 or Equation 34.

t1=to1-pΔt, t2=to2-pΔt
...(Number 33)
t1=to1+pΔt, t2=to2+pΔt
...(Equation 34) While increasing the value of p by 1, t obtained by Equation 33
1, t2 and t1, t2 obtained by equation 34
Repeat this process until the desired data is obtained. In order to perform calculations efficiently, when repeating the processes [4] and [5], k when tc=t0
Focusing on the value of o, processing when this value is close to 1 is performed preferentially.

Flowcharts of the algorithms for the above processing are shown in FIGS. 5, 6, and 7. The meanings of the newly used symbols in these figures are as follows. Let kb and kf be variables that store the value of ko. Let kstart be the value of ko when tc=t0. Let kend be the value of ko when td=t1. Set the maximum value that p can take, and let this be pmax. Let b and f be integer variables that control the flow of processing. Note that when the key frame at time to1 is the last key frame, to2 is not given and calculation for obtaining t2 is not performed.

In the algorithms shown in FIGS. 5, 6, and 7, first, focus is placed on the first key frame (
501). Next, set the time of the currently focused key frame to t0 (502), set the times of the adjacent key frames to t0, to1, to2 (503), and p=0, b=f
=1, kb=kf=φ (504), and the process proceeds to judgment 601. If b+f>1 (if 601 is YES), judgment 6
Otherwise (if 601 is NO), the process proceeds to step 507. If the judgment 601 is YES, b≠1(6
When 02 is NO), the process advances to step 701 and b=1(60
2 is YES), t1 and t2 are determined by equation 33 (603). If t0<t1 (604 is Y
If ES), proceed to process 605; otherwise (604
is NO), the process advances to step 507. Judgment 604 is YES
If so, the joint angles in the interval [t0, t1] are interpolated and corrected by repeating the processes [4] and [5] (605). Then, in the middle of processing 605, k
If o=φ does not become true and kend=1 (606 becomes Y
If ES), proceed to process 505; otherwise (606
is NO), proceed to decision 607. If kb≠φ, then (6
If 07 is YES), proceed to decision 608; otherwise (if 607 is NO), set kb=kstart (after performing 610) and proceed to process 701. Judgment 60
7 is YES, kstart≠φ and ksta
If rt is closer to 1 than kb (if 608 is YES), set kb = kstart (after performing 610) and proceed to step 701; otherwise (if 608 is NO), set b = 0 and then (609 (after that), the process proceeds to process 701.

In process 701, p=p+1,
The process advances to judgment 702. If p≦pmax (702 is YE
If S), the process proceeds to decision 703; otherwise (if 702 is NO), the process proceeds to process 507. If the judgment 702 is YES, if f≠1 (703 is NO), proceed to process 601, and if f=1 (703 is YES), calculate t1 and t2 by equation 34 (704), [4] and By repeating the process in [5], the joint angles in the interval [t0, t1] are interpolated and corrected (705). and,
If ko=φ does not become true during process 705 and kend=1 (if 706 is YES), proceed to process 505,
Otherwise (if 706 is NO), proceed to decision 707. If kf≠φ (if 707 is YES), judgment 708
otherwise (if 707 is NO) kf=k
After starting (after performing 710) Judgment 6
Proceed to 01. When the judgment 707 is YES, kstart
If ≠φ and kstart is closer to 1 than kf, then (
If 708 is YES) then set kb=kstart (after performing 710) and proceed to decision 601, otherwise (if 708 is NO) set f=0 and then (709
), the process proceeds to decision 601.

Processing 507 is a process performed when it is not possible to interpolate between key frames within the range of the given torque tolerance, and interactively interpolates the key frame at time to1 or the joint torque tolerance. Fix it. Processing 50
After step 7 is completed, the process advances to step 502.

Processing 505 is a process performed when key frames can be interpolated within the range of the given torque tolerance, and focuses on the next key frame. After processing 505 is completed, the process advances to judgment 506. If the current focus is on the last keyframe (if 506 is YES), end the process, otherwise (if 506 is NO) process 50
Proceed to step 2.

[7] Process Inclusion Relationship FIG. 8 shows the inclusion relationship of processes [2] to [6] constituting the present invention. Video generation processing of multi-joint structures (80
The outermost process of 1) is the process of correcting the time required between key frames (806) described in [6]. Processing 806
Among them, the interpolation process (804) of joint angles between key frames described in [4] and the correction process (805) of joint angles between key frames described in [5] are performed. Joint angular acceleration correction process (803) described in [3] in process 805
Do the following. In the process 803, the inverse dynamic calculation process (802) described in [2] is performed.

[0053] In the manner described above, data on joint angles of a multi-joint structure that passes through all key frames and moves within the allowable range of given joint torques can be obtained with a small amount of calculation. By using the method of the present invention, realistic movement of a multi-joint structure can be easily obtained.

[0054]

According to the present invention, it becomes possible to easily generate a realistic moving image of a multi-joint structure that takes mechanics into account from key frames.

[Brief explanation of the drawing]

FIG. 1 is a diagram explaining the concept of processing of the present invention.

FIG. 2 is a diagram illustrating an overview of processing of the present invention.

FIG. 3 is a system configuration diagram when the processing of the present invention is applied to animation production of a human body model.

FIG. 4 is a diagram showing a multi-jointed structure.

FIG. 5 is the first of three diagrams showing an algorithm for determining the time required between key frames.

FIG. 6 is the second of three diagrams showing an algorithm for determining the time required between key frames.

FIG. 7 is the third of three diagrams showing an algorithm for determining the time required between key frames.

FIG. 8 is a diagram showing the inclusion relationship of the processing of the present invention.

[Explanation of symbols]

101...Key frame at a certain time of the multi-joint structure, 102...Key frame next to key frame 101, 1
03...One of the joint angles in key frame 101, 10
4... Among the joint angles in key frame 102, those corresponding to 103, 105... Among the joint angles interpolated between key frames 101 and 102, those that start from 102 and end at 103, 106... Between key frames 101 and 102 Required time, 107...Of the joint torques between key frames 101 and 102, corresponding to 105, 108...1
07 Range of permissible joint torque values that limit 201...key frame data of multi-joint structure, 202...permissible joint torque value of multi-joint structure, 203...1 of multi-joint structure
Frame data, 204...Moving image generation program for multi-jointed structure.

Claims (2)

[Claims] 1. A key frame consisting of joint angle data at a certain time of a multi-joint structure having one or more joints;
A multi-jointed structure characterized in that the time required between adjacent key frames is determined from the joint torque tolerance value of each joint, and the result is used to obtain movement data for interpolating between key frames. video image generation method. 2. The method for calculating the motion data includes giving an initial value to the required time, interpolating the joint angle between the key frames using a cubic function, and calculating the joint torque by inverse dynamics calculation. To prevent joint torque from exceeding the above-mentioned allowable value,
2. The method of generating a moving image of a multi-joint structure according to claim 1, wherein the movement data is obtained by performing a correction by multiplying the joint angular acceleration by a real number and a correction by changing the required time.