Three-dimensional virtual character image generation method and device

文档序号：192771 发布日期：2021-11-02 浏览：29次中文

阅读说明：本技术 三维虚拟角色图像生成方法及装置 (Three-dimensional virtual character image generation method and device ) 是由梁彦军于 2021-08-05 设计创作，主要内容包括：本申请涉及一种三维虚拟角色图像生成方法及装置,涉及计算机视觉领域,该方法包括：使用预设的虚拟角色图像对目标人物图像的脸部区域进行替换,生成目标人物的第一二维虚拟角色图像；根据目标人物的二维虚拟角色图像,得到虚拟几何数据；将目标人物图像对应的纹理数据输入生成式对抗网络的生成器,生成虚拟纹理数据；基于虚拟几何数据和虚拟纹理数据,生成目标人物对应的三维虚拟角色图像。本申请能够成功地将目标人物图像转化为三维虚拟角色图像,使图像更加的生动形象,用户体验较好。(The application relates to a method and a device for generating a three-dimensional virtual character image, which relate to the field of computer vision, and the method comprises the following steps: replacing the face area of the target character image by using a preset virtual character image to generate a first two-dimensional virtual character image of the target character; obtaining virtual geometric data according to the two-dimensional virtual character image of the target character; inputting texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data; and generating a three-dimensional virtual character image corresponding to the target character based on the virtual geometric data and the virtual texture data. The method and the device can successfully convert the target character image into the three-dimensional virtual character image, so that the image is more vivid and the user experience is better.)

1. A method for generating a three-dimensional virtual character image, comprising:

replacing the face area of the target character image by using a preset virtual character image to generate a first two-dimensional virtual character image of the target character;

acquiring first geometric data and texture data corresponding to the target character image;

correcting the first geometric data according to the two-dimensional virtual character image of the target character to obtain virtual geometric data;

inputting texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data;

generating a three-dimensional virtual character to be rendered according to the virtual geometric data and the virtual texture data, and converting the three-dimensional virtual character into a second two-dimensional virtual character image;

and inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generating countermeasure network to obtain a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to the target character based on the virtual geometric data and the virtual texture data.

2. The method of claim 1, wherein modifying the first geometric data based on the two-dimensional avatar image of the target person to obtain virtual geometric data comprises:

extracting second geometric data from the first two-dimensional virtual character image of the target person;

extracting third geometric data corresponding to the second geometric data from the first geometric data, and performing two-dimensional data conversion processing on the third geometric data;

and correcting the first geometric data by using the second geometric data and the third geometric data after the two-dimensional data processing to obtain virtual geometric data.

3. The method according to claim 1, wherein after the first two-dimensional avatar image and the second two-dimensional avatar image are input to the discriminator of the generative confrontation network and the discrimination result output by the discriminator is obtained, the method further comprises:

if the judgment result is false, generating derived virtual texture data different from the virtual texture data through the generator;

generating a derivative three-dimensional virtual character image according to the derivative virtual texture data and the virtual geometric data, and converting the derivative three-dimensional virtual character image into a derivative two-dimensional virtual character image;

and comparing the derived two-dimensional virtual character image with the first two-dimensional virtual character image in the discriminator until the discrimination result is true.

4. The method of claim 1, wherein before inputting the first two-dimensional avatar image and the second two-dimensional avatar image into the discriminators of the generative confrontation network, the method further comprises:

and carrying out standardization processing on the first two-dimensional virtual character image and the second two-dimensional virtual character image.

5. The method of claim 1, wherein the generating the antagonistic network is trained by:

acquiring texture data corresponding to the target person image;

inputting texture data corresponding to the target character image into a generator for generating a countermeasure network, and outputting virtual texture data to be judged;

generating a three-dimensional virtual character to be judged according to the virtual texture data to be judged and the virtual geometric data, and converting the three-dimensional virtual character to be judged into a two-dimensional virtual character image to be judged;

inputting the two-dimensional virtual character image to be judged and the first two-dimensional virtual character image into a discriminator, and outputting a judgment result, wherein the discriminator comprises a preset number of convolutions, and the judgment result is used for indicating whether the two-dimensional virtual character image to be judged is consistent with the first two-dimensional virtual character image;

and adjusting parameters of the generated countermeasure network based on the discrimination result and the target function to obtain a virtual texture data generation model, wherein the virtual texture data generation model is used for generating virtual texture data corresponding to the target character.

6. The method of claim 5, wherein the objective function comprises: a pixel loss function, a perceptual loss function, and a style loss function;

the objective function is obtained according to the following method:

comparing the pixel data of the first two-dimensional virtual character image with the pixel data of the second two-dimensional virtual character image to obtain the pixel loss function;

and respectively obtaining each layer of convolution characteristics of the first two-dimensional virtual character image and the second two-dimensional virtual character image, and comparing each layer of convolution of the first two-dimensional virtual character image with each layer of convolution characteristics of the second two-dimensional virtual character image to obtain the perception loss function and the style loss function.

7. A three-dimensional virtual character image generating apparatus, comprising:

the two-dimensional image generation module is used for replacing the face area of the target character image by using a preset virtual character image to generate a first two-dimensional virtual character image of the target character;

the data acquisition module is used for acquiring first geometric data and texture data corresponding to the target character image;

the correction module is used for correcting the first geometric data according to the two-dimensional virtual character image of the target character to obtain virtual geometric data;

the data generation module is used for inputting texture data corresponding to the target character image into a generator of a generating type confrontation network to generate virtual texture data;

the image conversion module is used for generating a three-dimensional virtual character to be rendered according to the virtual geometric data and the virtual texture data and converting the three-dimensional virtual character into a second two-dimensional virtual character image;

and the three-dimensional image generation module is used for inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generating countermeasure network to obtain a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to the target character based on the virtual geometric data and the virtual texture data.

8. The apparatus of claim 7, wherein the correction module is further configured to:

extracting second geometric data from the first two-dimensional virtual character image of the target person;

extracting third geometric data corresponding to the second geometric data from the first geometric data, and performing two-dimensional data conversion processing on the third geometric data;

and correcting the first geometric data by using the second geometric data and the third geometric data after the two-dimensional data processing to obtain virtual geometric data.

9. The computer equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the method for generating a three-dimensional virtual character image according to any one of claims 1 to 6 when executing a program stored in the memory.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the three-dimensional virtual character image generation method according to any one of claims 1 to 6.

Technical Field

The present application relates to the field of computer vision, and in particular, to a method and an apparatus for generating a three-dimensional virtual character image.

Background

At present, functions of a special effect camera, property shooting and the like become standard matching of video App, a user can automatically synthesize personalized facial special effects with a specific style only by uploading a photo or a video, and can change the user into a cartoon figure, a portrait, an ancient figure and the like, but the functions are basically only limited in the stylization of two-dimensional images, and the generation of stylized three-dimensional images is very important in order to enable the user to change the user into the cartoon figure, the portrait, the ancient figure and the like which are more vivid and vivid, but the prior art cannot convert single pictures into three-dimensional virtual character images at present, and the user experience is poor.

Disclosure of Invention

In order to solve the technical problem or at least partially solve the technical problem, the application provides a three-dimensional virtual character image generation method and device.

In a first aspect, the present application provides a method for generating a three-dimensional virtual character image, including:

replacing the face area of the target character image by using a preset virtual character image to generate a first two-dimensional virtual character image of the target character;

acquiring first geometric data and texture data corresponding to a target character image;

correcting the first geometric data according to the two-dimensional virtual character image of the target character to obtain virtual geometric data;

inputting texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data;

and inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generative countermeasure network to obtain a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to the target character based on the virtual geometric data and the virtual texture data.

In a second aspect, the present application provides a three-dimensional virtual character image generating apparatus, including:

the data acquisition module is used for acquiring first geometric data and texture data corresponding to the target character image;

the correction module is used for correcting the first geometric data according to the two-dimensional virtual character image of the target character to obtain virtual geometric data;

the data generation module is used for inputting texture data corresponding to the target character image into a generator of the generative confrontation network to generate virtual texture data;

and the three-dimensional image generation module is used for inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generation type countermeasure network to obtain a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to the target character based on the virtual geometric data and the virtual texture data.

In a third aspect, an air conditioner control device is provided, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor configured to implement the steps of the method for generating a three-dimensional virtual character image according to any one of the embodiments of the first aspect when executing a program stored in the memory.

In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the three-dimensional virtual character image generation method as in any one of the embodiments of the first aspect.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:

according to the method provided by the embodiment of the application, the preset virtual character image is used for replacing the face area of the target character image, the first two-dimensional virtual character image of the target character is generated, and the two-dimensional virtual character image of the target character can be obtained. The method comprises the steps of obtaining first geometric data and texture data corresponding to a target character image, correcting the first geometric data according to a two-dimensional virtual character image of the target character to obtain virtual geometric data, inputting the texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data, and successfully obtaining the corrected virtual data of the target character. The method comprises the steps of generating a three-dimensional virtual character to be rendered according to virtual geometric data and virtual texture data, converting the three-dimensional virtual character into a second two-dimensional virtual character image, inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of a generating countermeasure network, obtaining a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to a target character based on the virtual geometric data and the virtual texture data.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic flowchart of a method for generating a three-dimensional virtual character image according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a method for generating a three-dimensional virtual character image according to another embodiment of the present application;

fig. 3 is a schematic flowchart of a method for generating a three-dimensional virtual character image according to yet another embodiment of the present application;

fig. 4 is a schematic structural diagram of a three-dimensional virtual character image generating apparatus according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a schematic flow chart of a method for generating a three-dimensional virtual character image according to an embodiment of the present application. As shown in fig. 1, the method includes:

step 101, replacing the face area of the target character image with a preset virtual character image to generate a first two-dimensional virtual character image of the target character.

In one embodiment of the present application, the target character image may be a photograph of the user, and the preset avatar image may be a two-dimensional stylized image corresponding to a face area in the photograph of the user, i.e., a first two-dimensional avatar image of the target character.

In specific implementation, for example, the target person image may be input into a pre-trained two-dimensional stylized image neural network, and the first two-dimensional virtual character image of the target person may be output.

The training process of the two-dimensional stylized image neural network is as follows:

obtaining sample data, the sample data comprising: the image processing method comprises the following steps of acquiring real person images of a plurality of persons and two-dimensional virtual character images corresponding to the real person images respectively, wherein the real person images can be acquired based on a CelebA-HQ data set;

and training a preset neural network by using the sample data to obtain a trained two-dimensional stylized image neural network.

And 102, acquiring first geometric data and texture data corresponding to the target character image.

In one embodiment of the present application, the first geometry data may be a predicted three-dimensional geometry of the target person, and the texture data may be a predicted three-dimensional texture map of the target person.

For the first geometric data and texture data obtaining method, for example:

based on the target figure image, identity, expression, texture, posture and illumination parameters of the three-dimensional face Model are predicted by using a 3D deformation statistical Model (3D digital media Model for short) coefficient regression neural network, then three-dimensional face reconstruction is carried out by using the predicted identity, expression, texture, posture and illumination parameters, and further first geometric data and texture data corresponding to the target figure image are predicted.

A specific example of the above-described acquisition of the first geometric data and texture data corresponding to the target person image is provided as follows:

predicting the shape and attitude coefficient (c) of a Basel Face Model (BFM for short) from a target person image by adopting a 3DMM coefficient regression neural network_i，c_e，c_t，p，γ)∈R²⁵⁷Wherein c is_i∈R⁸⁰,，c_e∈R⁶⁴，c_t∈R⁸⁰Respectively corresponding to the identity, expression and texture mapping coefficients of BFM, and p is belonged to R⁶，γ∈R²⁷Corresponding to the facial pose and the illumination coefficient of the target figure, and then performing three-dimensional face reconstruction by using the predicted identity, expression, texture, pose and illumination parameters so as to predict first geometric data S corresponding to the image of the target figure_rAnd texture data T_rThe method comprises the following steps:

wherein the content of the first and second substances,the face average shape (average geometry of faces in the large dataset),is the face average texture map (average texture map of the face in the big data set), B_id，B_exp，B_tPCA bases for identity, expression and texture mapping, respectively (i.e. statistical operations are performed on large data sets, PCA base is the statistical data).

And 103, correcting the first geometric data according to the two-dimensional virtual character image of the target person to obtain virtual geometric data.

In an embodiment of the present application, as shown in fig. 2, modifying the first geometric data according to the two-dimensional virtual character image of the target person to obtain virtual geometric data includes:

step 201, second geometric data is extracted from the first two-dimensional virtual character image of the target person.

In the present embodiment, the second geometric data extracted from the first two-dimensional virtual character image of the target person may be two-dimensional key point data of the target person, for example, it may be geometric data of key positions of the nose, glasses, ears, mouth, and the like of the target person.

Step 202, extracting third geometric data corresponding to the second geometric data from the first geometric data, and performing two-dimensional data conversion processing on the third geometric data.

In this embodiment, the third geometric data extracted from the first geometric data may be three-dimensional key point data corresponding to the two-dimensional key point data (the three-dimensional key point data is preset, and it is intuitively understood that the first point of the two-dimensional key point data is an eyeball, and then the first point of the three-dimensional key point data is also an eyeball). It should be noted that the two-dimensional key point data is of a virtual character, and the three-dimensional key point data is of a target person, and the purpose of the embodiment of the present application is to deform the geometric shape of the target person so that the point position projected onto the two-dimensional plane is consistent with the two-dimensional key point.

And 203, correcting the first geometric data by using the second geometric data and the third geometric data after the two-dimensional data processing to obtain virtual geometric data.

It should be understood that the three-dimensional key point data is projected onto the two-dimensional plane, so that the third geometric data (i.e. the first two-dimensional key point) after the two-dimensional data processing can be obtained, the second geometric data (i.e. the second two-dimensional key point) and the third geometric data after the two-dimensional data processing are compared, and the correction of the first geometric data can be completed according to the comparison result.

The following provides a specific example to illustrate the embodiments of the present application:

acquiring a first two-dimensional virtual character image of a target character;

inputting the first two-dimensional virtual character image of the target character into the trained key point prediction neural network, and outputting 68 two-dimensional key points l of the first two-dimensional virtual character image_s∈R⁶⁸；

From virtual geometric data S_SCorresponding 68 three-dimensional key points are taken out and projected into a two-dimensional space to obtain 68 deformed two-dimensional key points l'_s∈R⁶⁸。

Subsequently, the loss function of the key point is calculated as follows:

in order to make the virtual geometric data obtained after the correction smoother, a laplacian loss function may be used to limit the severe deformation of the geometric shape, which is specifically as follows:

where Lap (S, f) is the laplacian matrix of a three-dimensional face with geometry S and patch set f. It should be noted that S is a point set of the three-dimensional face, f is a patch set, a patch in the patch set is generally a closed region composed of three points, and the laplacian matrix is created according to the geometric shape S and the patch set f.

The loss function for virtual geometry data correction is:

where α is a loss function balance factor (which is empirically derived).

Finally, the virtual geometry data modified loss function is propagated back to the virtual geometry data S_SGuide virtual geometry data S_SCompleting the correction, namely:

wherein eta is a learning rate (a preset fixed value),is a parameter S_sOf the gradient of (c). It should be noted that the gradient is intended to be a vector (vector) indicating that the directional derivative of a certain function at the point takes the maximum value along the direction, i.e. the function changes the fastest and the maximum rate (modulo the gradient) along the direction (the direction of the gradient) at the point.

And 104, inputting texture data corresponding to the target character image into a generator of the generating type countermeasure network to generate virtual texture data.

In this embodiment, the generative countermeasure network is obtained by training, and the training method of the generative countermeasure network may be executed by the server or the terminal, which is not limited herein.

And 105, generating a three-dimensional virtual character to be rendered according to the virtual geometric data and the virtual texture data, and converting the three-dimensional virtual character into a second two-dimensional virtual character image.

In an embodiment of the present application, based on step 1045, in order to improve the generating effect of the virtual texture data, the objective function includes: a pixel loss function, a perceptual loss function, a style loss function, and a counter loss function;

the objective function is obtained according to the following method:

comparing the pixel data of the first two-dimensional virtual character image with the pixel data of the second two-dimensional virtual character image to obtain a pixel loss function;

and respectively obtaining the convolution characteristics of each layer of the first two-dimensional virtual character image and the second two-dimensional virtual character image, and comparing the convolution characteristics of each layer of the first two-dimensional virtual character image with the convolution characteristics of each layer of the second two-dimensional virtual character image to obtain a perception loss function and a style loss function.

In specific implementation, as an example:

using a micro-renderer to render virtual geometry data S_SAnd virtual texture data T_SIs rendered into a two-dimensional image I'_r；

Predicting a first two-dimensional virtual character image I using a trained face region segmentation neural network_rFace region ofBy segmenting out a first two-dimensional virtual character image I_rFace region ofThe interference of non-face areas (hair, background, etc.) can be eliminated, and the image processing effect is improved.

Using VGG19Neural network extraction of I separately_rAnd l'_rThe layers of convolution features of (a);

wherein, I_rAnd l'_rShould be similar, and therefore, a pixel loss function is introduced to measure I_rAnd l'_rIs represented as I_rAnd l'_rThe L1 distance between face region pixel values is as follows:

however, using pixel loss alone does not yield good high frequency information (image content, spatial structure, etc.) of the image, i.e., pixel loss does not measure I well_rAnd l'_rThus, embodiments of the present application introduce a perceptual loss function, contributing to l'_rHas a sum of_rSimilar high-level semantic features, which can more robustly measure the similarity of images during training, are represented as I_rAnd l'_rThe L1 distance between the convolution features of each layer is as follows:

wherein phi is_iIs the i-th layer convolution characteristic of the VGG19 neural network.

Further, in order to make I_rAnd l'_rWithout too much style difference (color, texture, etc.), embodiments of the present application also introduce a style loss function for measuring I_rAnd l'_rIs represented as I_rAnd l'_rThe L1 distance of the gram matrix of each layer convolution characteristic is as follows:

furthermore, to generate high quality stylized texture mapsT_sIs shown by_rAnd l'_rInput into a discriminator network D, the resulting penalty function is:

wherein the content of the first and second substances,for use in the constraint generator or generators of constraints,for constraining the arbiter. D (I)_r) Or D (I'_r) And may be 1 (true) or 0 (false). In this embodiment, as another implementation manner, the discriminator may be used only for calculating the loss function, and is not used in specific applications, that is, the discriminator is used only for the training generator, and after generating the virtual texture data, the three-dimensional virtual character image corresponding to the target person may be generated directly based on the virtual geometric data and the virtual texture data, without involving the discriminator.

In the embodiment of the application, the generation of the countermeasure network is trained through the following steps:

and 1041, acquiring texture data corresponding to the target person image.

In one embodiment of the present application, the texture data corresponding to the target person image may be generated in advance for matching with the virtual texture data.

Step 1042, inputting the texture data corresponding to the target character image into a generator for generating a countermeasure network, and outputting the virtual texture data to be determined.

In one embodiment of the present application, U-Net may be used as a generative countermeasure network, and the texture data corresponding to the target human image may be input into the generator, and the virtual texture data to be determined may be output.

And 1043, generating a three-dimensional virtual character to be judged according to the virtual texture data and the virtual geometric data to be judged, and converting the three-dimensional virtual character to be judged into a two-dimensional virtual character image to be judged.

It should be understood that, since the subsequent determiner performs the determination by comparing the image of the first two-dimensional virtual character with the image of the first two-dimensional virtual character, the three-dimensional virtual character to be determined needs to be converted into the image of the two-dimensional virtual character to be determined.

And step 1044, inputting the two-dimensional virtual character image to be judged and the first two-dimensional virtual character image into a discriminator, and outputting a judgment result, wherein the discriminator comprises a preset number of convolutions, and the judgment result is used for indicating whether the two-dimensional virtual character image to be judged is consistent with the first two-dimensional virtual character image.

In an embodiment of the present application, the discriminator receives two inputs, one is a two-dimensional virtual character image to be determined, and the other is a first two-dimensional virtual character image, and the two-dimensional virtual character image to be determined and the first two-dimensional virtual character image are input into the discriminator, so that a determination result can be obtained.

And 1045, adjusting parameters of the generated countermeasure network based on the discrimination result and the target function to obtain a virtual texture data generation model, wherein the virtual texture data generation model is used for generating virtual texture data corresponding to the target person.

Step 106, inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generative countermeasure network to obtain a discrimination result output by the discriminator,

in an embodiment, the input of the discriminator is a first two-dimensional virtual character image and a second two-dimensional virtual character image, and the first two-dimensional virtual character image and the second two-dimensional virtual character image are input into the discriminator of the generative countermeasure network, so as to obtain a discrimination result. If the judgment result is true, the first two-dimensional virtual character image is consistent with the second two-dimensional virtual character image, and at the moment, a three-dimensional virtual character image corresponding to the target character is generated based on the virtual geometric data and the virtual texture data, so that the three-dimensional virtual character image can be ensured to be more vivid.

In order to ensure that the second two-dimensional virtual character image has a high-quality processing effect, the first two-dimensional virtual character image and the second two-dimensional virtual character image are input into the discriminator of the generative countermeasure network, and after a discrimination result output by the discriminator is obtained, as shown in fig. 3, the method further includes:

step 301, if the result of the discrimination is false, generating derived virtual texture data different from the virtual texture data through a generator;

step 302, generating a derivative three-dimensional virtual character image according to the derivative virtual texture data and the virtual geometric data, and converting the derivative three-dimensional virtual character image into a derivative two-dimensional virtual character image;

and 303, comparing the derived two-dimensional virtual character image with the first two-dimensional virtual character image in a discriminator until the discrimination result is true.

Further, in order to ensure consistency of the first two-dimensional virtual character image and the second two-dimensional virtual character image during image processing and improve processing accuracy, before the first two-dimensional virtual character image and the second two-dimensional virtual character image are input into the discriminator of the generator countermeasure network, the method further includes:

and carrying out standardization processing on the first two-dimensional virtual character image and the second two-dimensional virtual character image.

Specifically, the formula of the normalization process is as follows:

Xa＝(X-μ)/b

wherein, X represents a first two-dimensional virtual character image, mu represents an average value of the first two-dimensional virtual character image and a second two-dimensional virtual character image, and b represents a variance of the first two-dimensional virtual character image and the second two-dimensional virtual character image.

As can be seen from the above, in the three-dimensional virtual character image generation method provided in the embodiment of the present application, the preset virtual character image is used to replace the face area of the target character image, so as to generate the first two-dimensional virtual character image of the target character, and thus the two-dimensional virtual character image of the target character can be obtained. The method comprises the steps of obtaining first geometric data and texture data corresponding to a target character image, correcting the first geometric data according to a two-dimensional virtual character image of the target character to obtain virtual geometric data, inputting the texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data, and successfully obtaining the corrected virtual data of the target character. The method comprises the steps of generating a three-dimensional virtual character to be rendered according to virtual geometric data and virtual texture data, converting the three-dimensional virtual character into a second two-dimensional virtual character image, inputting the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of a generating countermeasure network, obtaining a discrimination result output by the discriminator, and if the discrimination result is true, generating a three-dimensional virtual character image corresponding to a target character based on the virtual geometric data and the virtual texture data.

Based on the same inventive concept, the embodiment of the present invention further provides a three-dimensional virtual character image generating apparatus, as in the following embodiments. Because the principle of the three-dimensional virtual character image generation device for solving the problems is similar to the three-dimensional virtual character image generation method, the implementation of the three-dimensional virtual character image generation device can refer to the implementation of the three-dimensional virtual character image generation method, and repeated parts are not described again. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.

Fig. 4 is a schematic structural diagram of a three-dimensional virtual character image generating apparatus according to an embodiment of the present application, and as shown in fig. 4, the apparatus includes:

a two-dimensional image generation module 401, configured to replace a face area of a target character image with a preset virtual character image, and generate a first two-dimensional virtual character image of the target character;

a data obtaining module 402, configured to obtain first geometric data and texture data corresponding to a target person image;

a modification module 403, configured to modify the first geometric data according to the two-dimensional virtual character image of the target person, so as to obtain virtual geometric data;

a data generating module 404, configured to input texture data corresponding to the target character image into a generator of the generative countermeasure network, and generate virtual texture data;

an image conversion module 405, configured to generate a three-dimensional virtual character to be rendered according to the virtual geometric data and the virtual texture data, and convert the three-dimensional virtual character into a second two-dimensional virtual character image;

and a three-dimensional image generation module 406, configured to input the first two-dimensional virtual character image and the second two-dimensional virtual character image into a discriminator of the generative countermeasure network, obtain a discrimination result output by the discriminator, and if the discrimination result is true, generate a three-dimensional virtual character image corresponding to the target person based on the virtual geometric data and the virtual texture data.

In an alternative embodiment, the modification module 403 is further configured to:

extracting second geometric data from the first two-dimensional virtual character image of the target person;

extracting third geometric data corresponding to the second geometric data from the first geometric data, and performing two-dimensional data conversion processing on the third geometric data;

and correcting the first geometric data by using the second geometric data and the third geometric data after the two-dimensional data processing to obtain virtual geometric data.

In an alternative embodiment, the apparatus further comprises:

the data generation module generates derived virtual texture data different from the virtual texture data through the generator if the judgment result is false;

the image conversion module is used for generating a derivative three-dimensional virtual character image according to the derivative virtual texture data and the virtual geometric data and converting the derivative three-dimensional virtual character image into a derivative two-dimensional virtual character image;

and the comparison module is used for comparing the derived two-dimensional virtual character image with the first two-dimensional virtual character image in the discriminator until the discrimination result is true.

In an alternative embodiment, the apparatus further comprises:

and the image processing module is used for carrying out standardization processing on the first two-dimensional virtual character image and the second two-dimensional virtual character image.

In an alternative embodiment, the generation of the countermeasure network is trained by:

acquiring texture data and virtual texture data corresponding to the target character image;

inputting texture data corresponding to the target character image and virtual texture data into a generator for fusion, and outputting virtual texture data to be judged;

generating a three-dimensional virtual character to be judged according to the virtual texture data and the virtual geometric data to be judged, and converting the three-dimensional virtual character to be judged into a two-dimensional virtual character image to be judged;

inputting a two-dimensional virtual character image to be judged and a first two-dimensional virtual character image into a discriminator and outputting a judgment result, wherein the discriminator comprises a preset number of convolutions, and the judgment result is used for indicating whether the two-dimensional virtual character image to be judged is consistent with the first two-dimensional virtual character image;

In an alternative embodiment, the objective function includes: a pixel loss function, a perceptual loss function, and a style loss function;

the objective function is obtained according to the following method:

comparing the pixel data of the first two-dimensional virtual character image with the pixel data of the second two-dimensional virtual character image to obtain a pixel loss function;

Fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention, where the computer device 500 shown in fig. 5 includes: at least one processor 501, memory 502, at least one network interface 504, and other user interfaces 503. The various components in the computer device 500 are coupled together by a bus system 505. It is understood that the bus system 505 is used to enable connection communications between these components. The bus system 505 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 505 in FIG. 5.

The user interface 503 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, trackball, touch pad, or touch screen, among others.

It is to be understood that the memory 502 in embodiments of the present invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (ddr Data Rate SDRAM, ddr SDRAM), Enhanced Synchronous SDRAM (ESDRAM), synchlronous SDRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

In some embodiments, memory 502 stores elements, executable units or data structures, or a subset thereof, or an expanded set thereof as follows: an operating system 5021 and application programs 5022.

The operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application 5022 includes various applications, such as a Media Player (Media Player), a Browser (Browser), and the like, for implementing various application services. The program for implementing the method according to the embodiment of the present invention may be included in the application program 5022.

In the embodiment of the present invention, by calling a program or an instruction stored in the memory 502, specifically, a program or an instruction stored in the application 5022, the processor 501 is configured to execute the method steps provided by the method embodiments, for example, including:

replacing the face area of the target character image by using a preset virtual character image to generate a first two-dimensional virtual character image of the target character;

acquiring first geometric data and texture data corresponding to the target character image;

correcting the first geometric data according to the two-dimensional virtual character image of the target character to obtain virtual geometric data;

inputting texture data corresponding to the target character image into a generator of a generating type countermeasure network to generate virtual texture data;

In an optional embodiment, the modifying the first geometric data according to the two-dimensional virtual character image of the target person to obtain virtual geometric data includes:

extracting second geometric data from the first two-dimensional virtual character image of the target person;

extracting third geometric data corresponding to the second geometric data from the first geometric data, and performing two-dimensional data conversion processing on the third geometric data;

and correcting the first geometric data by using the second geometric data and the third geometric data after the two-dimensional data processing to obtain virtual geometric data.

In an optional embodiment, after the first two-dimensional avatar image and the second two-dimensional avatar image are input to the discriminator of the generative confrontation network and the discrimination result output by the discriminator is obtained, the method further includes:

if the judgment result is false, generating derived virtual texture data different from the virtual texture data through the generator;

and comparing the derived two-dimensional virtual character image with the first two-dimensional virtual character image in the discriminator until the discrimination result is true.

In an optional embodiment, before the first two-dimensional avatar image and the second two-dimensional avatar image are input to the discriminator of the generative confrontation network, the method further comprises:

and carrying out standardization processing on the first two-dimensional virtual character image and the second two-dimensional virtual character image.

In an alternative embodiment, the generative confrontation network is trained by:

acquiring texture data corresponding to the target character image and the virtual texture data;

inputting texture data corresponding to the target character image and the virtual texture data into the generator for fusion, and outputting virtual texture data to be judged;

In an alternative embodiment, the objective function comprises: a pixel loss function, a perceptual loss function, and a style loss function;

the objective function is obtained according to the following method:

comparing the pixel data of the first two-dimensional virtual character image with the pixel data of the second two-dimensional virtual character image to obtain the pixel loss function;

And sending prompt information of successful placement of the plurality of target objects to a user. The method disclosed by the above-mentioned embodiments of the present invention may be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 501. The Processor 501 may be a general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software elements in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the method in combination with the hardware.

It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the Processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units configured to perform the functions described herein, or a combination thereof.

For a software implementation, the techniques described herein may be implemented by means of units performing the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

The computer device provided in this embodiment may be a computer device as shown in fig. 5, and may perform all the steps of the three-dimensional virtual character image generation method shown in fig. 1 to 3, so as to achieve the technical effect of the three-dimensional virtual character image generation method shown in fig. 1 to 3, and please refer to the description related to fig. 1 to 3, which is not repeated herein for brevity.

The embodiment of the invention also provides a storage medium (computer readable storage medium). The storage medium herein stores one or more programs. Among others, the storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, a hard disk, or a solid state disk; the memory may also comprise a combination of memories of the kind described above.

When one or more programs in the storage medium are executable by one or more processors, the three-dimensional virtual character image generation method described above is implemented.

The processor is configured to execute a three-dimensional virtual character image generation program stored in the memory to implement the steps of:

a three-dimensional virtual character image generation method comprises the following steps: