阅读说明：本技术 骨骼动画顶点矫正方法及模型的学习方法、装置和设备 (Skeleton animation vertex correction method and model learning method, device and equipment ) 是由 杨涛 唐浩 于 2021-09-22 设计创作，主要内容包括：本申请提供一种骨骼动画顶点矫正方法及模型的学习方法、装置和设备,包括采集不同人的包括T型姿态的各种动作的全身影像和姿态数据获得模型训练数据：标准3D网格模型的姿态数据P和P-(m)、N个顶点坐标Gr,T型姿态3D网格模型的N个顶点坐标T、n个关节点坐标J,构建并训练关节位置预测模型,构建姿态回归网络模型和包括标准蒙皮函数、训练好的关节位置预测模型和姿态回归网络模型的骨骼动画顶点矫正模型并训练。本申请提供的骨骼动画顶点矫正模型可以对人物角色在运动过程中原有的皮肤表面产生的不合理的形变进行补偿修正,使得人物角色看起来逼真、自然,提高了动画质量,减少了动画师的工作量,使得制作同样质量的动画花费的时间更少。(The application provides a skeleton animation vertex correction method and a model learning method, device and equipment, which comprises the steps of acquiring whole-body images and posture data of various actions including T-shaped postures of different people to obtain model training data: pose data P and P of a standard 3D mesh model m The method comprises the following steps of establishing and training a joint position prediction model, establishing a posture regression network model and a skeleton animation vertex correction model comprising a standard skin function, the trained joint position prediction model and the trained posture regression network model, and training, wherein the N vertex coordinates Gr and the N vertex coordinates T, N joint point coordinates J of the T-shaped posture 3D grid model are used for establishing and training a joint position prediction model. The skeleton animation vertex correction model provided by the application can compensate and correct unreasonable deformation generated on the original skin surface of the character in the motion process, so that the character angleThe color looks vivid and natural, the animation quality is improved, the workload of an animator is reduced, and the time for making the animation with the same quality is shorter.)

1. A learning method of a skeleton animation vertex correction model is characterized by comprising the following steps:

s11: collecting whole body images and posture data of different people including various actions of T-shaped postures, generating a standard 3D grid model with N vertexes and a T-shaped posture 3D grid model, and acquiring a posture data axial angle form P of the standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mN vertex coordinates Gr of the standard 3D grid model and N vertex coordinates T of the T-shaped posture 3D grid model, wherein N is a positive integer;

s12: joint skeleton binding is carried out on the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer;

s13: constructing a Joint position prediction model Joint _ net, wherein the output dimensionality of the Joint position prediction model is the same as the dimensionality of N Joint point coordinates of the T-shaped posture 3D model, taking N vertex coordinates T of the T-shaped posture 3D grid model as the input of the Joint position prediction model, and taking N Joint point coordinates J of the T-shaped posture 3D grid model as labels of a training Joint position prediction model to train the Joint position prediction model, so as to obtain the trained Joint position prediction model;

s14: constructing a posture regression network model Pose _ net, wherein the output dimensionality of the posture regression network model is the same as the dimensionality of N vertex coordinates of the standard 3D grid model, and the input of the posture regression network model is a posture data rotation matrix form P of the standard 3D grid model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

2. The learning method of the skeleton animation vertex correction model according to claim 1, wherein the step S11 includes:

s111: acquiring whole-body images of different people including various actions of T-shaped postures through a camera matrix, and synchronously acquiring posture data through a movable capturing device;

s112: establishing a 3D point cloud model by a stereoscopic vision matching method according to the whole body images of different people including various actions of T-shaped postures, and registering and aligning the obtained 3D point cloud model by combining a geometric constraint and image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N;

s113: establishing a coordinate system by taking human pelvis points as a coordinate origin, performing data normalization on the standard 3D grid model to obtain N vertex coordinates Gr of the standard 3D grid model, and performing data normalization on the T-shaped posture 3D grid model to obtain N vertex coordinates T of the T-shaped posture 3D grid model;

s114: establishing a one-to-one matching relation between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P of the standard 3D grid model, and converting the attitude data axial angle form P of the standard 3D grid model into a rotation matrix form P_m。

3. The learning method of the skeleton animation vertex correction model according to claim 1, wherein the step S13 includes:

s131: constructing a Joint position prediction model Joint _ net, wherein the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

s132: taking a tensor of converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as input of a joint position prediction model, and taking a tensor of converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label of a training joint position prediction model to train the joint position prediction model;

s133: optimizing a joint position prediction model by adopting a mean square loss function, wherein the output of the joint position prediction model is J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]And (3) a vector formed by coordinates of the ith Joint point which is output by the Joint position prediction model prediction is represented, and finally the trained Joint position prediction model J = Joint _ net (T) is obtained.

4. The learning method of an animated skeleton vertex correction model according to claim 1, wherein the step S14 comprises:

s141: constructing a posture regression network model Pose _ net, wherein the input data dimension of the posture regression network model is N x 3, and the output data dimension is N x 3;

s142: constructing a skeleton animation top by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ netPoint correction modelWhere V represents the set of vertex coordinates of the skeletal animation vertex correction model prediction output, and V = [ V ]₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mRepresenting the attitude data axis angle form and the rotation matrix form, Pose _ net (P), respectively, of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

s143: converting N vertex coordinates T of T-shaped posture 3D grid model into tensor T_inConverting the attitude data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minThe method comprises the steps of training a skeleton animation vertex correction model by taking N vertex coordinates Gr of a standard 3D mesh model as input of the skeleton animation vertex correction model and taking the N vertex coordinates Gr as labels for training the skeleton animation vertex correction model;

s144: optimizing the skeleton animation vertex correction model by adopting a loss function comprising the loss of calculating the vertex coordinate difference of the standard 3D mesh model and the loss of the weight W of the skeleton animation vertex correction model, wherein the mean square loss function for calculating the loss of the vertex coordinate difference of the standard 3D mesh model is as follows:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of the weight W of the skeleton animation vertex correction model isWhere loss _ w represents the skinning weight loss value, w_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows:and the loss represents the total loss value of the skeleton animation vertex correction model, represents the weight occupied by the loss of the vertex coordinate difference of the 3D mesh model, and finally obtains the trained skeleton animation vertex correction model.

5. A bone animation vertex correction method is characterized by comprising the following steps:

acquiring posture data and a T-shaped posture 3D grid model of a human body;

inputting the posture data of the human body and the T-shaped posture 3D mesh model into the trained skeleton animation vertex correction model to obtain N vertex coordinates of the standard 3D mesh model of the human body in the posture;

wherein the skeleton animation vertex correction model is trained by adopting the learning method of the skeleton animation vertex correction model as claimed in any one of claims 1 to 4.

6. A learning device of a skeleton animation vertex correction model is characterized by comprising:

a training data acquisition module (31) for acquiring the whole body images and posture data of various actions including T-shaped postures of different people, generating a standard 3D grid model and a T-shaped posture 3D grid model with N vertexes, and acquiring a posture data axial angle form P of the standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mN vertex coordinates Gr of the standard 3D grid model and N vertex coordinates T of the T-shaped posture 3D grid model, wherein N is a positive integer;

the joint data acquisition module (32) is used for binding joint bones of the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer;

the Joint model obtaining module (33) is used for constructing a Joint position prediction model Joint _ net, the output dimensionality of the Joint position prediction model is the same as the dimensionality of N Joint point coordinates of the T-shaped posture 3D model, N vertex coordinates T of the T-shaped posture 3D grid model are used as the input of the Joint position prediction model, N Joint point coordinates J of the T-shaped posture 3D grid model are used as labels of a training Joint position prediction model to train the Joint position prediction model, and the trained Joint position prediction model is obtained;

a vertex model obtaining module (34) for constructing a posture regression network model Pose _ net, the output dimensionality of the posture regression network model is the same as the dimensionality of N vertex coordinates of the standard 3D grid model, and the input of the posture regression network model is a posture data rotation matrix form P of the standard 3D grid model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

7. The learning device of a skeletal animation vertex correction model according to claim 6, wherein the training data acquisition module (31) comprises:

the human body data acquisition unit is used for acquiring whole body images of different people with various actions including T-shaped gestures through a camera matrix and synchronously acquiring gesture data through the mobile capture equipment;

the 3D model acquisition unit is used for establishing a 3D point cloud model through a stereoscopic vision matching method according to the whole body images of various actions including T-shaped postures of different people, and registering and aligning the obtained 3D point cloud model by combining a geometric constraint and image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N;

a vertex coordinate obtaining unit, configured to establish a coordinate system with the human pelvis point as a coordinate origin, and perform data normalization on the standard 3D mesh model to obtain N vertex coordinates Gr, Gr = [ g ] of the standard 3D mesh model₀,g₁,g₂,…g_(N-1)]，g_iRepresenting the 3-dimensional coordinate value of each vertex of the standard 3D grid model, and carrying out data normalization on the T-shaped posture 3D grid model to obtain N vertex coordinates T of the T-shaped posture 3D grid model, wherein T = [ T ]₀,t₁,t₂,…t_(N-1)] ，t_i3-dimensional coordinate values of each vertex of the T-shaped posture 3D grid model are represented;

the attitude data matching unit is used for establishing a one-to-one matching relationship between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P of the standard 3D grid model, and converting the axial angle form P of the attitude data of the standard 3D grid model into a rotation matrix form P_m。

8. The learning device of a skeletal animation vertex correction model according to claim 6, wherein the joint model acquisition module (33) comprises:

the Joint model building unit is used for building a Joint position prediction model Joint _ net, the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

the joint model training unit is used for taking a tensor formed by converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as the input of the joint position prediction model, and taking a tensor formed by converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label for training the joint position prediction model to train the joint position prediction model;

a joint model optimization unit for optimizing a joint position prediction model with a mean square loss function, the joint position prediction model having an output of J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]To representVector formed by the real coordinates of the i-th joint point, J_pre[i]And (3) a vector formed by coordinates of the ith Joint point which is output by the Joint position prediction model prediction is represented, and finally the trained Joint position prediction model J = Joint _ net (T) is obtained.

9. The learning device of a skeletal animation vertex correction model according to claim 6, wherein the vertex model obtaining module (34) comprises:

the attitude model building unit is used for building an attitude regression network model Pose _ net, the input data dimension of the attitude regression network model is N x 3, and the output data dimension is N x 3;

a vertex model construction unit for constructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ netWhere V represents the set of vertex coordinates of the skeletal animation vertex correction model prediction output, and V = [ V ]₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mAxial angle form and rotation matrix form, Pose _ net (P), respectively representing Pose data of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

a vertex model training unit for converting the N vertex coordinates T of the T-shaped posture 3D grid model into tensor T_inConverting the attitude data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minInputting the vertex correction model of the skeletal animation, and using the N vertex coordinates Gr of the standard 3D mesh model as the vertex correction of the training skeletal animationTraining the skeleton animation vertex correction model by using the label of the positive model;

a vertex model optimization unit for optimizing the skeleton animation vertex correction model by using a loss function including a loss of calculating a standard 3D mesh model vertex coordinate difference and a loss of a weight W of the skeleton animation vertex correction model, the mean square loss function of calculating the loss of the standard 3D mesh model vertex coordinate difference being:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of weight W of the skeleton animation vertex correction model is as follows, wherein loss _ W represents a skin weight loss value, W_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows:and the loss represents the total loss value of the skeleton animation vertex correction model, represents the weight occupied by the loss of the vertex coordinate difference of the 3D mesh model, and finally obtains the trained skeleton animation vertex correction model.

10. A computer device, comprising:

a memory for storing a computer program;

a processor for executing said computer program for implementing the steps of the learning method of the skeletal animated vertex correction model according to any one of claims 1 to 5.

Technical Field

The application relates to the technical field of character skinning, in particular to a learning method of a skeleton animation vertex correction model, a learning device of the skeleton animation vertex correction model, a skeleton animation vertex correction method and computer equipment.

Background

At present, AR intelligent technology is rapidly developing, content intake requirements of Internet users are more abundant, requirements for animation characters, AR and other contents are more intense, and in skeleton animation, after character roles complete skeleton binding, mapping brush weights from the character skeleton to grid vertexes are needed, so that reasonable deformation is formed on the surfaces of the animation characters in the motion process, and the character role covering is achieved. The traditional skeleton animation skin is completely made by manpower, different parts of different characters need to be manually endowed with different weights, and the process is called a weight brushing process.

In the process of covering the character, for different character characters and different parts, an animator needs to give different weights by repeatedly observing the surface deformation in the character motion process, the workload is very complicated, and meanwhile, the final output animation is difficult to be ensured to be consistent with the expected performance, such as the problems of collapse, skin surface shriveling at the joint in a linear hybrid covering model and the like. The animation engineer is caused to repeatedly reset the weight of the role skinning, so that the quality and the output efficiency of the animation are greatly reduced, and the cost of the animation is high. The existing method of IK constraint solves the problem of skin collapse and solves the weight through human tissue dynamics, but the methods have the problems of large workload, long calculation time, inaccurate result, dependence on the experience of an animator on the final effect and the like, and cannot be widely applied.

Disclosure of Invention

The application aims to provide a learning method of a skeleton animation vertex correction model, a learning device of the skeleton animation vertex correction model, a skeleton animation vertex correction method and computer equipment, which can compensate and correct unreasonable deformation generated on the original skin surface of a character in the motion process, and avoid the problems of skin surface collapse, distortion and the like of the character in certain postures in the motion process. The specific scheme is as follows:

the application provides a learning method of a skeleton animation vertex correction model, which comprises the following steps:

s11: collecting whole body images and posture data of different people including various actions of T-shaped postures, generating a standard 3D grid model with N vertexes and a T-shaped posture 3D grid model, and acquiring a posture data axial angle form P of the standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mN vertex coordinates Gr of the standard 3D grid model and N vertex coordinates T of the T-shaped posture 3D grid model, wherein N is a positive integer;

s12: joint skeleton binding is carried out on the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer;

s13: constructing a Joint position prediction model Joint _ net, wherein the output dimensionality of the Joint position prediction model is the same as the dimensionality of N Joint point coordinates of the T-shaped posture 3D model, taking N vertex coordinates T of the T-shaped posture 3D grid model as the input of the Joint position prediction model, and taking N Joint point coordinates J of the T-shaped posture 3D grid model as labels of a training Joint position prediction model to train the Joint position prediction model, so as to obtain the trained Joint position prediction model;

s14: constructing a posture regression network model Pose _ net, wherein the output dimensionality of the posture regression network model is the same as the dimensionality of N vertex coordinates of the standard 3D grid model, and the input of the posture regression network model is a posture data rotation matrix form P of the standard 3D grid model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

Optionally, the S11 includes:

s111: acquiring whole-body images of different people including various actions of T-shaped postures through a camera matrix, and synchronously acquiring posture data through a movable capturing device;

s112: establishing a 3D point cloud model by a stereoscopic vision matching method according to the whole body images of different people including various actions of T-shaped postures, and registering and aligning the obtained 3D point cloud model by combining a geometric constraint and image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N;

s113: establishing a coordinate system by taking human pelvis points as a coordinate origin, performing data normalization on the standard 3D grid model to obtain N vertex coordinates Gr of the standard 3D grid model, and performing data normalization on the T-shaped posture 3D grid model to obtain N vertex coordinates T of the T-shaped posture 3D grid model;

s114: establishing a one-to-one matching relation between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P of the standard 3D grid model, and converting the attitude data axial angle form P of the standard 3D grid model into a rotation matrix form P_m。

Optionally, the S13 includes:

s131: constructing a Joint position prediction model Joint _ net, wherein the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

s132: taking a tensor of converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as input of a joint position prediction model, and taking a tensor of converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label of a training joint position prediction model to train the joint position prediction model;

s133: optimizing a joint position prediction model by adopting a mean square loss function, wherein the output of the joint position prediction model is J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]A vector formed by coordinates of the ith Joint point which represents the prediction output of the Joint position prediction model is obtained, and finally the trained Joint position prediction model J = Joint _ n is obtainedet(T)。

Optionally, the S14 includes:

s141: constructing a posture regression network model Pose _ net, wherein the input data dimension of the posture regression network model is N x 3, and the output data dimension is N x 3;

s142: constructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ netWhere V represents the set of vertex coordinates of the skeletal animation vertex correction model prediction output, and V = [ V ]₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mRepresenting the attitude data axis angle form and the rotation matrix form, Pose _ net (P), respectively, of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

s143: converting N vertex coordinates T of T-shaped posture 3D grid model into tensor T_inConverting the attitude data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minThe method comprises the steps of training a skeleton animation vertex correction model by taking N vertex coordinates Gr of a standard 3D mesh model as input of the skeleton animation vertex correction model and taking the N vertex coordinates Gr as labels for training the skeleton animation vertex correction model;

s144: optimizing the skeleton animation vertex correction model by adopting a loss function comprising the loss of calculating the vertex coordinate difference of the standard 3D mesh model and the loss of the weight W of the skeleton animation vertex correction model, wherein the mean square loss function for calculating the loss of the vertex coordinate difference of the standard 3D mesh model is as follows:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of the weight W of the skeleton animation vertex correction model isWhere loss _ w represents the skinning weight loss value, w_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows:and the loss represents the total loss value of the skeleton animation vertex correction model, represents the weight occupied by the loss of the vertex coordinate difference of the 3D mesh model, and finally obtains the trained skeleton animation vertex correction model.

The application also provides a bone animation vertex correction method, which comprises the following steps:

acquiring posture data and a T-shaped posture 3D grid model of a human body;

inputting the posture data of the human body and the T-shaped posture 3D mesh model into the trained skeleton animation vertex correction model to obtain N vertex coordinates of the standard 3D mesh model of the human body in the posture;

the skeleton animation vertex correction model is trained by adopting the learning method of the skeleton animation vertex correction model.

The present application further provides a learning device for a skeleton animation vertex correction model, including:

a training data obtaining module 31, configured to collect whole body images and posture data of various actions including T-shaped postures of different people, generate a standard 3D mesh model and a T-shaped posture 3D mesh model having N vertices, and obtain a posture data axis angle form P of the standard 3D mesh model and a posture data rotation matrix form P of the standard 3D mesh model_mN vertex coordinates Gr of the standard 3D grid model and N vertex coordinates T of the T-shaped posture 3D grid model, wherein N is a positive integer;

the joint data acquisition module 32 is used for binding joint bones of the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer;

the Joint model obtaining module 33 is configured to construct a Joint position prediction model Joint _ net, an output dimension of the Joint position prediction model is the same as a dimension of N Joint point coordinates of the T-type posture 3D model, N vertex coordinates T of the T-type posture 3D mesh model are used as an input of the Joint position prediction model, and N Joint point coordinates J of the T-type posture 3D mesh model are used as a label for training the Joint position prediction model to train the Joint position prediction model, so that a trained Joint position prediction model is obtained;

a vertex model obtaining module 34, configured to construct a Pose regression network model Pose _ net, where an output dimension of the Pose regression network model is the same as dimensions of N vertex coordinates of the standard 3D mesh model, and an input of the Pose regression network model is a Pose data rotation matrix form P of the standard 3D mesh model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

Optionally, the training data obtaining module 31 includes:

the human body data acquisition unit is used for acquiring whole body images of different people with various actions including T-shaped gestures through a camera matrix and synchronously acquiring gesture data through the mobile capture equipment;

the 3D model acquisition unit is used for establishing a 3D point cloud model through a stereoscopic vision matching method according to the whole body images of various actions including T-shaped postures of different people, and registering and aligning the obtained 3D point cloud model by combining a geometric constraint and image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N;

a vertex coordinate obtaining unit, configured to establish a coordinate system with the human pelvis point as a coordinate origin, and perform data normalization on the standard 3D mesh model to obtain N vertex coordinates Gr, Gr = [ g ] of the standard 3D mesh model₀,g₁,g₂,…g_(N-1)]，g_iRepresenting the 3-dimensional coordinate value of each vertex of the standard 3D grid model, and carrying out data normalization on the T-shaped posture 3D grid model to obtain N vertex coordinates T of the T-shaped posture 3D grid model, wherein T = [ T ]₀,t₁,t₂,…t_(N-1)] ，t_i3-dimensional coordinate values of each vertex of the T-shaped posture 3D grid model are represented;

the attitude data matching unit is used for establishing a one-to-one matching relationship between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P of the standard 3D grid model, and converting the axial angle form P of the attitude data of the standard 3D grid model into a rotation matrix form P_m。

Optionally, the joint model obtaining module 33 includes:

the Joint model building unit is used for building a Joint position prediction model Joint _ net, the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

the joint model training unit is used for taking a tensor formed by converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as the input of the joint position prediction model, and taking a tensor formed by converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label for training the joint position prediction model to train the joint position prediction model;

a joint model optimization unit for optimizing a joint position prediction model with a mean square loss function, the joint position prediction model having an output of J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]Representing joint position prediction model prediction inputAnd finally obtaining a trained Joint position prediction model J = Joint _ net (T) by using a vector formed by coordinates of the ith Joint point.

Optionally, the vertex model obtaining module 34 includes:

the attitude model building unit is used for building an attitude regression network model Pose _ net, the input data dimension of the attitude regression network model is N x 3, and the output data dimension is N x 3;

a vertex model construction unit, configured to construct a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net, and a posture regression network model Pose _ net, where V represents a set of vertex coordinates output by the skeleton animation vertex correction model prediction, and V = [ V ], [₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mAxial angle form and rotation matrix form, Pose _ net (P), respectively representing Pose data of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

a vertex model training unit for converting the N vertex coordinates T of the T-shaped posture 3D grid model into tensor T_inConverting the attitude data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minThe method comprises the steps of training a skeleton animation vertex correction model by taking N vertex coordinates Gr of a standard 3D mesh model as input of the skeleton animation vertex correction model and taking the N vertex coordinates Gr as labels for training the skeleton animation vertex correction model;

a vertex model optimization unit for optimizing the skeleton animation vertex correction model by using a loss function including a loss of calculating a standard 3D mesh model vertex coordinate difference and a loss of a weight W of the skeleton animation vertex correction model, the mean square loss function of calculating the loss of the standard 3D mesh model vertex coordinate difference being:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of the weight W of the skeleton animation vertex correction model isWhere loss _ w represents the skinning weight loss value, w_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows:and the loss represents the total loss value of the skeleton animation vertex correction model, represents the weight occupied by the loss of the vertex coordinate difference of the 3D mesh model, and finally obtains the trained skeleton animation vertex correction model.

The present application further provides a computer device, comprising:

a memory for storing a computer program;

a processor for executing the computer program to implement the steps of the learning method of the skeleton animation vertex correction model as described above.

The application provides a learning method of a skeleton animation vertex correction model, which comprises the steps of acquiring full-body images and posture data of various actions including T-shaped postures of different people to obtain posture data P of a standard 3D grid model and posture data rotation matrix form P of the standard 3D grid model_mThe method comprises the steps of obtaining N vertex coordinates Gr of a standard 3D grid model, N vertex coordinates T of a T-shaped posture 3D grid model, binding joint bones of the T-shaped posture 3D grid model to obtain N joint point coordinates J of the T-shaped posture 3D grid model, building and training a joint position prediction model, building a posture regression network model, building and training a skeleton animation vertex correction model comprising a standard skin function, the trained joint position prediction model and the posture regression network model, and obtaining and training a skeleton animation vertex correction model comprising the standard skin function, the trained joint position prediction model and the posture regression network modelAnd (5) training a bone animation vertex correction model.

Therefore, the skeleton animation vertex correction model provided by the application can compensate and correct the unreasonable deformation generated on the original skin surface of the character in the motion process, so that the character looks vivid, natural and reasonable in the motion process, the animation quality is improved, the workload of an animator in the animation post-production process is reduced, the time spent on producing the animation with the same quality is shorter, and the cost is lower.

The application also provides a learning device of the skeleton animation vertex correction model, a skeleton animation vertex correction method and computer equipment, and the learning device, the skeleton animation vertex correction method and the computer equipment all have the beneficial effects and are not repeated.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flowchart illustrating a learning method of a skeleton animation vertex correction model according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a method for acquiring training data according to an embodiment of the present disclosure;

fig. 3 is a flowchart illustrating a method for obtaining a joint position prediction model according to an embodiment of the present disclosure;

fig. 4 is a schematic network structure diagram of a joint position prediction model according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for obtaining a skeleton animation vertex correction model according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of a network structure of a posture regression network model according to an embodiment of the present application

FIG. 7 is a schematic flowchart illustrating a method for correcting a vertex of a skeletal animation according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of a learning apparatus for a skeleton animation vertex correction model according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the scope of protection of the present application.

In the process of covering the character, for different character characters and different parts, an animator needs to give different weights by repeatedly observing the surface deformation in the character motion process, the workload is very complicated, and meanwhile, the final output animation is difficult to be ensured to be consistent with the expected performance, such as the problems of collapse, skin surface shriveling at the joint in a linear hybrid covering model and the like. The animation engineer is caused to repeatedly reset the weight of the role skinning, so that the quality and the output efficiency of the animation are greatly reduced, and the cost of the animation is high. The existing method of IK constraint solves the problem of skin collapse and solves the weight through human tissue dynamics, but the methods have the problems of large workload, long calculation time, inaccurate result, dependence on the experience of an animator on the final effect and the like, and cannot be widely applied.

Based on the above problems, the present embodiment provides a learning method for a skeleton animation vertex correction model, where the skeleton animation vertex correction model constructed based on a standard skinning function, a trained joint position prediction model and a posture regression network model can compensate and correct unreasonable deformation generated on the original skin surface of a character in a motion process, so that the character appears vivid, natural and reasonable in the motion process, the animation quality is improved, and meanwhile, the workload of an animator in an animation post-production process is reduced, so that the time spent on producing animations with the same quality is less, and the cost is lower.

Referring to fig. 1 specifically, fig. 1 is a method for learning a skeleton animation vertex correction model according to an embodiment of the present application, which specifically includes:

s11: collecting whole body images and posture data of different people including various actions of T-shaped postures, generating a standard 3D grid model with N vertexes and a T-shaped posture 3D grid model, and acquiring a posture data axial angle form P of the standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mThe three-dimensional model comprises N vertex coordinates Gr of a standard 3D grid model and N vertex coordinates T of a T-shaped posture 3D grid model, wherein N is a positive integer.

In one implementation, as shown in fig. 2, S11 includes:

s111: the whole-body images of different people including various actions of T-shaped postures are collected through a camera matrix, and meanwhile posture data are synchronously collected through a movable capturing device.

S112: according to the whole body images of different people including various actions of T-shaped postures, a 3D point cloud model is established through a stereoscopic vision matching method, and the obtained 3D point cloud model is registered and aligned by combining a geometric constraint method and an image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N.

S113: establishing a coordinate system by taking human pelvis points as a coordinate origin, and performing data normalization on the standard 3D grid model to obtain N vertex coordinates Gr of the standard 3D grid model, wherein Gr = [ g ]₀,g₁,g₂,…g_(N-1)]，g_iRepresenting the 3-dimensional coordinate value of each vertex of the standard 3D grid model, and carrying out data normalization on the T-shaped posture 3D grid model to obtain N vertex coordinates T of the T-shaped posture 3D grid model, wherein T = [ T ]₀,t₁,t₂,…t_(N-1)] ，t_iAnd 3-dimensional coordinate values of each vertex of the T-shaped posture 3D grid model are represented.

S114: establishing a one-to-one matching relation between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P, P = [ P ] of the standard 3D grid model₀,p_1, p_2,…p_(n-1)]，p_iRepresenting Euler rotation angle of ith joint point of human skeleton relative to father node, converting shaft angle form P of attitude data of standard 3D grid model into rotation matrix form P_m，P_m=[p⁰ _m, p¹ _m,…, p^n-1 _m]，pⁱ _mAnd the rotation matrix of the ith joint point of the human skeleton relative to the parent node is represented.

In the present embodiment, whole-body images (shape data) and posture data of different persons are acquired using a camera array and a motion capture device, and are generated and registered as a standard grid. The annular array of the camera surrounds the shot collection object, the specific form of the camera array is not limited in the embodiment, and the camera array can be designed according to actual requirements. For example, in the embodiment, 48 cameras are arranged in an annular manner, the cameras are divided into 16 columns, 3 cameras in each column are vertically arranged at intervals of 0.7 m, the 16 columns of cameras uniformly surround the studio for one circle to achieve 360-degree dead-angle-free coverage on the human body, and the cameras must have a synchronous acquisition function to ensure that the acquired images can achieve synchronous alignment of each image and parameter calibration is performed on the cameras before data acquisition each time.

In this embodiment, the camera array is used to collect data of various movements of the human body (mainly including daily movement movements of the human body such as running, walking, squatting, dancing, etc.), and since the cameras adopt the frame synchronization technology, each frame of data between the cameras can be matched one by one. Each frame of data collected by the camera needs to store a time stamp which is accurate to millisecond. Then, a 3D point cloud model is established through a stereo vision matching method, and the obtained 3D point cloud model is registered and aligned by combining a geometric constraint and an image projection constraint method to generate a standard 3D mesh model with a fixed number N of vertices. Then, a coordinate system is established by taking the human pelvis point as the origin of coordinates (the directions of coordinate axes can be random, but need to be consistent front and back), and the standard is matchedThe 3D grid model is subjected to data normalization to obtain a three-dimensional coordinate value Gr = [ g ] of each vertex₀,g₁,g₂,…g_(N-1)]，g_i=[x,y,z]Representing 3-dimensional coordinate values of each vertex of the 3D model, acquiring T-type attitude data of each person in a T-Pose (T-type attitude) state to generate a T-type attitude 3D mesh model, normalizing the coordinates of each T-type attitude 3D mesh model to obtain three-dimensional coordinate values T = [ T ] of each vertex₀,t₁,t₂,…t_(N-1)]；t_i=[x,y,z]Representing the three-dimensional space coordinates of the vertices of the T-shaped pose 3D mesh model, T ∈ R^N×3T _ psoe has a unified attitude data P = [ P ]₀,p*₁,p*₂,...p*_n-1]This amounts to a constant.

Specifically, before data is collected, the time of the movable capture device is adjusted to be consistent with the time of the camera, the frame rate of the movable capture data is set to be more than twice of the image collection frame rate of the camera, a timestamp of each frame of data is required to be stored when the movable capture device stores the data, and the timestamp of each frame of data can be stored when the movable capture device stores the data when the data is collected. After the acquisition is finished, comparing the time stamp of the dynamic capture data with the time stamp of the image data acquired by the camera array, and when the difference value of the two time stamps is smaller than a set threshold (the set threshold is 4 ms), considering that the dynamic capture data and the image data are acquired simultaneously to obtain a piece of matching data, so that the dynamic capture attitude motion data and a standard 3D grid model form a one-to-one matching relation, and simultaneously obtaining dynamic capture data P = [ P ] data₀,p₁,p₂, …p_n-1]Pi represents the Euler rotation angle of the ith joint point of the human skeleton relative to the father node, p_i=[θ₁，θ₂，θ₃]，θ₁，θ₂，θ₃Respectively representing a pitch angle, a yaw angle and a rolling angle (the sequence can be arbitrary, and the front and the back are consistent). Normalizing the rotation angle of the pose data P: p = [ P ]₀,p_1, p_2,…p_(n-1)]Handle P_jConversion into rotation matrix P by standard rotation angle conversion formula^j _m，P^j _m∈R^3×3，P_m=[p⁰ _m, p¹ _m,…, p^n-1 _m]，p^j _mA rotation matrix representing the jth joint point of the human skeleton relative to the parent node.

S12: and (3) carrying out joint skeleton binding on the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer.

In this embodiment, an animator can perform high-precision joint skeleton binding on a T-Pose 3D mesh model obtained under T _ dose by using a maya tool and derive joint point coordinates J, J = [ J ])₀,j₁,j₂,…j_(n-1)]Each T-type pose 3D grid model has n joint points, each joint contains x, y and z coordinates, and J belongs to R^n×3。

S13: and constructing a Joint position prediction model Joint _ net, wherein the output dimensionality of the Joint position prediction model is the same as the dimensionality of N Joint point coordinates of the T-shaped posture 3D model, taking N vertex coordinates T of the T-shaped posture 3D grid model as the input of the Joint position prediction model, and taking N Joint point coordinates J of the T-shaped posture 3D grid model as a label for training the Joint position prediction model to train the Joint position prediction model, so as to obtain the trained Joint position prediction model.

In one implementation, as shown in fig. 3, S13 includes:

s131: constructing a Joint position prediction model Joint _ net, wherein the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

s132: taking a tensor of converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as input of a joint position prediction model, and taking a tensor of converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label of a training joint position prediction model to train the joint position prediction model;

s133: optimizing a joint position prediction model by adopting a mean square loss function, wherein the output of the joint position prediction model is J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]And (3) a vector formed by coordinates of the ith Joint point which is output by the Joint position prediction model prediction is represented, and finally the trained Joint position prediction model J = Joint _ net (T) is obtained.

In this embodiment, a Joint position prediction model Joint _ net is constructed, as shown in fig. 4, where Input: n × 3 represents the input data dimension as N × 3, FC and ReLu represent the fully-connected layer and ReLu-activated layer of the neural network, respectively, the middle number represents the dimension of the Output data of each layer of the neural network, Output: n x 3 denotes the output data dimension is n x 3. Setting the input of Joint _ net to be Nx 3 (the registered 3D grid model has N vertexes, each vertex contains x, y and z coordinates), setting the output to be nx3 (each 3D grid model has N Joint points, each Joint contains x, y and z coordinates), using the N vertex coordinates T of the T-type posture 3D grid model obtained in the step S11 as the input, using the N Joint point coordinates J of the T-type posture 3D grid model as labels, using the mean square error function constraint to carry out the standardized learning training of the Joint position prediction model on the Joint _ net, and finally enabling the Joint position prediction model to predict the Joint coordinates of the T-type posture 3D grid model of any human body after the training convergence. I.e., J = Joint _ net (t).

The method comprises the following specific steps: (1) and constructing a Joint position prediction model Joint _ net, wherein the model structure is shown in figure 4. Joint _ net is a neural network with an input of N × 3 and an output of N × 3 (the data dimension of the T-type pose 3D mesh model established under T _ position is N × 3, and the dimension of Joint data J derived after bone binding is N × 3). (2) The N vertex coordinates T of the T-shaped posture 3D grid model are changed into an Nx 3 tensor, namely the N vertex coordinates T are spliced into an Nx 3 tensor which is used as the input of a Joint position prediction model Joint _ net, the N Joint point coordinates J of the T-shaped posture 3D grid model are changed into an N x 3 tensor, namely the N Joint point coordinates J are spliced into an N x 3 tensor which is used as a guide label for training the Joint position prediction model Joint _ net. (3) For Joint position prediction model Joint _ netThe mean square loss function is the optimization target, and the output of the network is J_preThe mean square loss calculation formula is as follows: where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]And a vector composed of coordinates of the i-th joint point which represents the prediction output of the joint position prediction model. (4) Setting a training parameter of a Joint position prediction model Joint _ net, wherein the specific parameter in this embodiment is as follows: the learning rate is 0.001, the optimization method is Adam (adaptive time estimation method), the number of iterations is 100, and the Batch size is 8. (5) And training the Joint _ net by adopting a standard neural network training method to finally realize J = Joint _ net (T).

By constructing a Joint position prediction model Joint _ net, the coordinates of each Joint of any character role under T _ Point can be correctly predicted, the skeleton of the character role can be automatically bound through a simple program if the coordinate value of each Joint exists, the time spent on the skeleton binding of the character role is much, and the manufacturing time, the cost and the difficulty of the character role can be greatly reduced by the technology of the application.

S14: constructing a posture regression network model Pose _ net, wherein the output dimensionality of the posture regression network model is the same as the dimensionality of N vertex coordinates of the standard 3D grid model, and the input of the posture regression network model is a posture data rotation matrix form P of the standard 3D grid model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

In one implementation, as shown in fig. 5, S14 includes:

s141: constructing a posture regression network model Pose _ net, wherein the input data dimension of the posture regression network model is N x 3, and the output data dimension is N x 3;

s142: and constructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, wherein V represents a set of vertex coordinates output by the skeleton animation vertex correction model in a prediction mode, and V = [ V ], [₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mAxial angle form and rotation matrix form, Pose _ net (P), respectively representing Pose data of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

s143: converting N vertex coordinates T of T-shaped posture 3D grid model into tensor T_inConverting the attitude data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minThe method comprises the steps of training a skeleton animation vertex correction model by taking N vertex coordinates Gr of a standard 3D mesh model as input of the skeleton animation vertex correction model and taking the N vertex coordinates Gr as labels for training the skeleton animation vertex correction model;

s144: optimizing the skeleton animation vertex correction model by adopting a loss function comprising the loss of calculating the vertex coordinate difference of the standard 3D mesh model and the loss of the weight W of the skeleton animation vertex correction model, wherein the mean square loss function for calculating the loss of the vertex coordinate difference of the standard 3D mesh model is as follows:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]Vertex correction model for representing skeleton animationMeasuring the coordinate vector of the ith vertex; a loss function for calculating the loss of weight W of the skeleton animation vertex correction model is as follows, wherein loss _ W represents a skin weight loss value, W_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows:and the loss represents the total loss value of the skeleton animation vertex correction model, represents the weight occupied by the loss of the vertex coordinate difference of the 3D mesh model, and finally obtains the trained skeleton animation vertex correction model.

In this embodiment, a posture regression network model Pose _ net is constructed, as shown in fig. 6, where Input: n × 3 × 3 represents the input data dimension as n × 3 × 3, FC and ReLu represent the fully-connected layer and the RuLu active layer of the neural network, respectively, the middle number represents the dimension of the Output data of each layer of the neural network, Output: n × 3 indicates an output data dimension of N × 3. The input to set the Pose _ net is nx3 × 3 (the registered 3D mesh model has N joint points, each joint point is represented by a rotation matrix and then is a 3 × 3 matrix), the output is nx3 (the registered 3D mesh model has N vertices, each vertex contains x, y, and z coordinates), and the final vertex coordinates V are calculated using a standard skinning function, where V is ═ V₀,v₁,v₂,…,v_N-1]。

In this embodiment, a skinning function is used, the embodiment does not specifically limit the type of the skinning function, and the method is applicable to any skinning algorithm in principle.

Specifically, a method of combining a linear skinning function (LBS) with a posture regression network model, Pose _ net, is used to improve the standard skinning function in combination with the posture regression network model, specifically, the standard linear skinning formula is V ═ LBS (T, P, J, W, P), where,i.e. where v is_iCoordinate vector representing the ith vertex, w_i,jAnd expressing the action weight of the j joint to the i vertex in the skin function, wherein the formula means that: the vertex of the 3D model in the current pose changes with respect to the rotation in the T-pose, and the meaning of the formula is: calculate the rotation of the jth joint point relative to the root node at the current pose, P, P_mRespectively representing the form of the shaft angle and the form of a rotation matrix of the attitude data of the standard 3D grid model, J representing the coordinates of n joint points of the T-shaped attitude 3D grid model, P^j _mThe posture data of j-th joint point in the character movement process is expressed in the form of rotation matrix, g_iAnd 3-dimensional coordinate values representing the ith vertex of the 3D character at T _ position. P represents the posture data of the T-shaped posture, is a constant, A (j) represents a path set formed by the shortest paths from the joint nodes to the root nodes, and is an ordered vector set from the root nodes to the nodes, wherein W = [ W = ]_0,0,w_0,1…w_0,n;w_1,0, w_1,1…w_1,n-1, … w_N-1,n-1]，w_i,jRepresenting the action weight of the jth joint point of the 3D role model in the linear skin model on the ith vertex for w_i,jComprises the following steps: 0<w_i,j<1 and (b).

In this embodiment, a calculation formula of using a linear skin function in combination with a Pose regression network model Pose _ net is as follows:

；

in (d), since J can be derived from J = Joint _ net (t), the formula can be written as:namely:wherein W is the weight of the skin，P^j _mIs attitude data P_jIs represented by a rotation matrix of 3 x 3 (the rotation of each joint point is represented by a rotation matrix of 3 x 3), P_m=[p⁰ _m, p¹ _m,…, p^n-1 _m]And setting W as a variable, so that in the process of training the skeleton animation vertex correction model, W becomes a variable parameter, and the skin function after improvement is a trainable neural network. Since the Joint position prediction model Joint _ net is trained, the Joint _ net parameter should be set to be non-conductive in the training LBS process, so that the Joint _ net parameter remains unchanged in the training LBS process.

Wherein, in the embodiment, each bit W of the weight value W in the LBS is improved_i,jSetting the skin function combined with Pose _ net as a neural network model, namely a skeleton animation vertex correction model, as a learnable variable, training the skeleton animation vertex correction model by using a neural network standardized training method, and optimizing w through the training and learning of the skeleton animation vertex correction model_i,jAnd (4) parameters. T _ pos model data T = [ T ]₀,t₁,t₂,…t_(N-1)]And corresponding rotation matrix representation form data P of attitude data_m∈R^n×3×3As the input of the vertex correction model of the skeleton animation, the N vertex coordinates Gr of the standard 3D mesh model, Gr = [ g ] are used₀,g₁,g₂,…g_(N-1)]As a label for training the skeleton animation vertex correction model. In the improved model, the role of the posture regression network model Pose _ net is equivalent to predicting the correction value v of each vertex of the 3D model under each posture data P for each posture data_ｓTherefore, a part of the shortage of the LBS is compensated.

Specifically, the acquired 3D model data T = [ T ] of T _ pos₀,t₁,t₂,…t_(N-1)]Constructing tensor Tin and tensor P constructed by attitude data P_inAnd converting P into a representation form P of a rotation matrix_mConstructed tensor P_minAs input data, Gr is used as a label, and a loss function is designed in two parts, namely: standard of meritLoss of the vertex coordinate difference of the 3D mesh model and loss of the weight W of the improved linear skin model (skeleton animation vertex correction model), and the skin model is optimized and improved by adopting a standard neural network learning method. Loss of the standard 3D mesh model vertex coordinate difference using the mean square loss function, Gr = [ g =₀,g₁,g₂,…g_(N-1)]The output of the skeletal animation vertex correction model V = [ V ] for N vertex coordinates of the standard 3D mesh model₀, v₁…, v_N-1]Calculating the mean square loss function of the loss of the vertex coordinate difference of the standard 3D mesh model as follows:where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of weight W of the skeleton animation vertex correction model is as follows, wherein loss _ W represents a skin weight loss value, W_i,jAnd representing the action weight of the j-th joint on the i-th vertex in the skinning function. In the training phase, each value of W is monitored before updating the parameters for each iteration, and W = [ W =_0,0, w_0,1…w_0,n;w_1,0, w_1,1…w_1,n-1 … w_N-1,n-1]W in_i,jWhen w is_i,j<At 0, w is set artificially_i,j=0, when w_i,j>At 1, w is set artificially_i,jAnd = 1. The combined loss function is:and the loss represents the total loss value of the skeleton animation vertex correction model and the weight of the loss of the vertex coordinate difference of the 3D mesh model. Setting training parameters for improving the LBS, the specific parameters in this embodiment are: the learning rate is 0.001, the optimization method is Adam (adaptive time estimation method), the number of iterations is 100, the Batch size is 8, and = 0.7. Training the skeleton animation vertex correction model by adopting a standard neural network training method to finally obtain the trained skeleton animation vertex correction modelAnd (4) modeling.

The advantages of the present application include: the method has the advantages that the camera rectangular array is used for collecting and reconstructing the 3D high-dimensional grid curved surface of the human body, the posture regression network model learns the deformation rule of the surface of the human body caused by the action by the machine learning method, the skin model combined with the posture regression network model learns the proper weight value by the method, the posture regression network model can output the proper skin weight value by using the Pose _ net technology, the complicated weight brushing process is omitted in the animation manufacturing process, the workload of practitioners is greatly reduced, the accuracy is improved, the less time is spent to obtain higher animation quality, the time is saved, and the animation manufacturing cost is reduced.

In addition, the method outputs the offset of one vertex to an improved skinning function through a vertex correction model of the skeletal animation, leads the vertex correction model of the skeletal animation to learn the deformation rule of the skin surface of the human body under different actions, can compensate and correct the unreasonable deformation generated on the original skin surface of the character in the motion process through the automatic Joint _ net of the character skeleton and the automatic Pose _ net of the character vertex compensation and repair technology, avoids the problems of skin surface collapse, distortion and the like of the character in certain postures in the motion process, thereby avoiding the unreasonable deformation, leading the character to look vivid, natural and reasonable in the motion process, improving the animation quality, reducing the workload of an animator in the post-production process of the animation, leading the production of the animation with the same quality and less time spent by using the method, the cost is lower.

The present embodiment further provides a method for correcting a vertex of a bone animation, as shown in fig. 7, specifically including:

s21: acquiring posture data and a T-shaped posture 3D grid model of a human body;

s22: inputting the posture data of the human body and the T-shaped posture 3D mesh model into the trained skeleton animation vertex correction model to obtain N vertex coordinates of the standard 3D mesh model of the human body in the posture.

In this embodiment, the trained skeletal animation is savedNetwork parameters of the vertex correction model, loading parameters into the bone animated vertex correction model V = LBS (T _ + pos _ net (Pm _), P _, Joint _ net (T _), W, P _) and keeping all parameters fixed, where W is the learned skinning weight after training. Making a T-shaped posture 3D grid model T \undera person T _ Point, providing any reasonable human skeleton posture data P \, converting P _ into an expression form Pm _ of a rotation matrix as an input variable to be input into a skeleton animation vertex correction model, namely V = LBS (T _ + Point _ net (Pm _), P _, Joint _ net (T _), W, P _), and obtaining V = [ V _ [ ] [ V _ ] ]₀, v₁…, v_N-1]The space coordinates of the vertexes of the human standard 3D mesh model in the posture are obtained.

The following describes a learning apparatus for a skeleton animation vertex correction model according to an embodiment of the present application, and the learning apparatus for a skeleton animation vertex correction model described below and the learning method for a skeleton animation vertex correction model described above may be referred to in correspondence.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a learning apparatus for a skeleton animation vertex correction model according to an embodiment of the present application, which specifically includes:

a training data obtaining module 31, configured to collect whole body images and posture data of various actions including T-shaped postures of different people, generate a standard 3D mesh model and a T-shaped posture 3D mesh model having N vertices, and obtain a posture data axis angle form P of the standard 3D mesh model and a posture data rotation matrix form P of the standard 3D mesh model_mN vertex coordinates Gr of the standard 3D grid model and N vertex coordinates T of the T-shaped posture 3D grid model, wherein N is a positive integer;

the joint data acquisition module 32 is used for binding joint bones of the T-shaped posture 3D grid model to obtain n joint point coordinates J of the T-shaped posture 3D grid model, wherein n is a positive integer;

the Joint model obtaining module 33 is configured to construct a Joint position prediction model Joint _ net, an output dimension of the Joint position prediction model is the same as a dimension of N Joint point coordinates of the T-type posture 3D model, N vertex coordinates T of the T-type posture 3D mesh model are used as an input of the Joint position prediction model, and N Joint point coordinates J of the T-type posture 3D mesh model are used as a label for training the Joint position prediction model to train the Joint position prediction model, so that a trained Joint position prediction model is obtained;

a vertex model obtaining module 34, configured to construct a Pose regression network model Pose _ net, where an output dimension of the Pose regression network model is the same as dimensions of N vertex coordinates of the standard 3D mesh model, and an input of the Pose regression network model is a Pose data rotation matrix form P of the standard 3D mesh model_mConstructing a skeleton animation vertex correction model by combining a standard skin function, a trained Joint position prediction model Joint _ net and a posture regression network model Pose _ net, and rotating a posture data axial angle form P of a standard 3D grid model and a posture data rotation matrix form P of the standard 3D grid model_mAnd taking N vertex coordinates T of the T-shaped posture 3D mesh model as input of the skeleton animation vertex correction model, and taking N vertex coordinates Gr of the standard 3D mesh model as labels for training the skeleton animation vertex correction model to train the skeleton animation vertex correction model, so as to obtain the trained skeleton animation vertex correction model.

Preferably, the training data acquisition module 31 includes:

the human body data acquisition unit is used for acquiring whole body images of different people with various actions including T-shaped gestures through a camera matrix and synchronously acquiring gesture data through the mobile capture equipment;

the 3D model acquisition unit is used for establishing a 3D point cloud model through a stereoscopic vision matching method according to the whole body images of various actions including T-shaped postures of different people, and registering and aligning the obtained 3D point cloud model by combining a geometric constraint and image projection constraint method to generate a standard 3D grid model and a T-shaped posture 3D grid model with a fixed vertex number N;

a vertex coordinate obtaining unit, configured to establish a coordinate system with the human pelvis point as a coordinate origin, and perform data normalization on the standard 3D mesh model to obtain N vertex coordinates Gr, Gr = [ g ] of the standard 3D mesh model₀,g₁,g₂,…g_(N-1)]，g_iExpressing the 3-dimensional coordinate value of each vertex of the standard 3D grid model, and performing data normalization on the T-shaped posture 3D grid model to obtain the T-shaped postureN vertex coordinates T, T = [ T ] of 3D mesh model₀,t₁,t₂,…t_(N-1)] ，t_i3-dimensional coordinate values of each vertex of the T-shaped posture 3D grid model are represented;

the attitude data matching unit is used for establishing a one-to-one matching relationship between the acquired attitude data and the standard 3D grid model to obtain an attitude data axial angle form P of the standard 3D grid model, and converting the axial angle form P of the attitude data of the standard 3D grid model into a rotation matrix form P_m。

Preferably, the joint model acquisition module 33 includes:

the Joint model building unit is used for building a Joint position prediction model Joint _ net, the input data dimension of the Joint position prediction model is N x 3, and the output data dimension is N x 3;

the joint model training unit is used for taking a tensor formed by converting N vertex coordinates T of the T-shaped posture 3D grid model into N x 3 as the input of the joint position prediction model, and taking a tensor formed by converting N joint point coordinates J of the T-shaped posture 3D grid model into N x 3 as a label for training the joint position prediction model to train the joint position prediction model;

a joint model optimization unit for optimizing a joint position prediction model with a mean square loss function, the joint position prediction model having an output of J_preThe mean square loss calculation formula is:where loss represents the mean square loss, J [ i ]]Vector formed by the real coordinates of the ith joint point, J_pre[i]And (3) a vector formed by coordinates of the ith Joint point which is output by the Joint position prediction model prediction is represented, and finally the trained Joint position prediction model J = Joint _ net (T) is obtained.

Preferably, the vertex model obtaining module 34 includes:

the attitude model building unit is used for building an attitude regression network model Pose _ net, the input data dimension of the attitude regression network model is N x 3, and the output data dimension is N x 3;

a vertex model construction unit for combining the standard skin function and trainingConstructing a skeleton animation vertex correction model by using a Joint position prediction model Joint _ net and a posture regression network model Pose _ net, wherein V represents a set of vertex coordinates predicted and output by the skeleton animation vertex correction model, and V = [ V ]₀, v₁…, v_N-1]，v_iCoordinate vector representing ith vertex, LBS linear skin function, T vertex coordinate of T-pose 3D mesh model, P, P_mAxial angle form and rotation matrix form, Pose _ net (P), respectively representing Pose data of a standard 3D mesh model_m) Representing for each pose P_mPredicting a vertex correction value, wherein Joint _ net (T) represents that n Joint point coordinates under the human T-shaped posture are predicted for each T-shaped posture, W represents the weight of a linear skin function, P represents the posture data of the T-shaped posture, W is set as a conductible parameter, and Joint _ net is set as an un-conductible parameter;

a vertex model training unit for converting N vertex coordinates T of the T-shaped posture 3D grid model into tensor Tin and converting posture data P of the standard 3D grid model into tensor P_in、P_mInto a tensor P_minThe method comprises the steps of training a skeleton animation vertex correction model by taking N vertex coordinates Gr of a standard 3D mesh model as input of the skeleton animation vertex correction model and taking the N vertex coordinates Gr as labels for training the skeleton animation vertex correction model;

a vertex model optimization unit for optimizing the skeleton animation vertex correction model by using a loss function including a loss of calculating a standard 3D mesh model vertex coordinate difference and a loss of a weight W of the skeleton animation vertex correction model, the mean square loss function of calculating the loss of the standard 3D mesh model vertex coordinate difference being: where loss _ v represents the loss value of the difference in the calculated vertex coordinates, G [ i]Coordinate vector representing the ith vertex of the standard 3D model, V [ i ]]A coordinate vector representing the ith vertex predicted by the skeleton animation vertex correction model; a loss function for calculating the loss of weight W of the skeleton animation vertex correction model is as follows, wherein loss _ W represents a skin weight loss value, W_i,jAnd representing the action weight of the j joint in the skinning function to the i vertex, wherein the combined loss function is as follows: wherein loss represents the total loss value of the skeleton animation vertex correction model and represents the vertex coordinate difference of the 3D mesh modelThe loss of the skeleton animation takes up weight, and finally the trained skeleton animation vertex correction model is obtained.

Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.

In the following, a computer device provided by the embodiments of the present application is introduced, and the computer device described below and the method described above may be referred to correspondingly.

The present application further provides a computer device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the learning method of the skeleton animation vertex correction model as described above.

Since the embodiment of the computer device portion and the embodiment of the method portion correspond to each other, please refer to the description of the embodiment of the method portion for the embodiment of the computer device portion, which is not repeated here.

The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The method for learning a skeleton animation vertex correction model, the device for learning a skeleton animation vertex correction model, the method for correcting a skeleton animation vertex, and the computer device provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.