阅读说明：本技术 一种基于生成对抗网络的点云三维重建方法与系统 (Point cloud three-dimensional reconstruction method and system based on generation countermeasure network ) 是由 王若梅 卢林鹏 周凡 林淑金 林格 于 2021-08-19 设计创作，主要内容包括：本发明公开了一种基于生成对抗网络的点云三维重建方法与系统。包括：首先处理等待重建的点云数据,其次搭建生成器模块和判别器模块,然后对生成器和判别器进行多次对抗训练得到训练完成后的生成器,最终训练完成的生成器生成的就是等待重建的点云数据对应的三维模型。本发明以已经拥有的点云数据为基准最终得到相对应的三维模型,能够应用于多种多样的单个物体,并且在多个模型的不同部分的拼接任务中能够生成较为光滑的过度区域,而不会很出现非常突兀的变化,其次损失函数的设计更好地加快网络模型的收敛速度,体现点云数据的三维特性,最终生成的对应三维模型精确度更高。(The invention discloses a point cloud three-dimensional reconstruction method and system based on a generated countermeasure network. The method comprises the following steps: firstly, processing point cloud data waiting for reconstruction, secondly, building a generator module and a discriminator module, then carrying out a plurality of confrontation training on the generator and the discriminator to obtain a generator after the training is finished, and finally, generating a three-dimensional model corresponding to the point cloud data waiting for reconstruction by the generator after the training is finished. The method and the device take the existing point cloud data as a reference to finally obtain the corresponding three-dimensional model, can be applied to various single objects, can generate smooth transition areas in splicing tasks of different parts of a plurality of models without very abrupt changes, and then the design of a loss function can better accelerate the convergence speed of a network model, embody the three-dimensional characteristics of the point cloud data, and finally generate the corresponding three-dimensional model with higher accuracy.)

1. A point cloud three-dimensional reconstruction method based on a generation countermeasure network is characterized by comprising the following steps:

processing point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed;

building a generator module, inputting the three-dimensional grid model, obtaining the displacement of each vertex in the model, adding the displacement and the vertex coordinates of the three-dimensional grid model to obtain a new three-dimensional grid model, and uniformly sampling some points from the new model to obtain point cloud data generated by the generator;

constructing a discriminator module, inputting the point cloud data waiting for reconstruction and the point cloud data generated from the generator, and outputting a probability value smaller than 1;

training and optimizing the discriminator and the generator for multiple times to obtain a trained discriminator and a trained generator;

and finally, the generator after training generates a three-dimensional model corresponding to the point cloud data to be reconstructed.

2. The method for point cloud three-dimensional reconstruction based on generation of countermeasure network as claimed in claim 1, wherein said processing of point cloud data waiting for reconstruction results in a three-dimensional mesh model surrounding the point cloud data waiting for reconstruction, specifically:

carrying out normalization processing on point cloud data to be reconstructed, and normalizing the coordinates of each point to be between-1 and 1;

if the three-dimensional model to be obtained is formed by splicing different parts of a plurality of three-dimensional models, point clouds of the parts to be spliced are required to be placed at corresponding positions, and normalization processing is also carried out;

and (4) using three-dimensional modeling software to obtain a three-dimensional grid model capable of surrounding the point cloud data after the normalization treatment.

3. The method for reconstructing a point cloud based on a generated countermeasure network as claimed in claim 1, wherein the generator module is built, the input is the three-dimensional mesh model, the displacement of each vertex in the model is obtained, the displacement is added to the vertex coordinates of the three-dimensional mesh model to obtain a new three-dimensional mesh model, some points are uniformly sampled from the new model to obtain the point cloud data generated by the generator, specifically:

the generator adopts MeshCNN, is a neural network acting on each edge in the three-dimensional mesh, and firstly calculates five characteristics possessed by each edge according to the three-dimensional mesh, wherein the five characteristics are dihedral angles between two triangular patches where each edge is located, and the ratios of the height of opposite angles in the two triangular patches where the edge is located and the height of the two triangular patches where the edge is located to the edge;

the calculated characteristic is similar to the convolution neural network of the image and is sent to the network for calculation, so that the displacement required by each vertex in the three-dimensional grid is obtained;

adding the displacement of the point and the vertex coordinates of the three-dimensional mesh model to obtain a three-dimensional mesh model closer to real point cloud data;

and uniformly sampling some points from the new model to obtain point cloud data generated by the generator.

4. The method for reconstructing a point cloud in three dimensions based on generation of a countermeasure network as claimed in claim 1, wherein the input of the discriminator building module is the point cloud data waiting for reconstruction and the point cloud data generated from the generator, and the output is a probability value smaller than 1, specifically:

the discriminator is a two-classifier, a PointNet + + network model is adopted as the discriminator, and the point cloud data waiting for reconstruction and the point cloud data generated from the generator are input;

modifying the output number of the PointNet + + full connection to be 1, judging whether the point cloud data waiting for reconstruction is obtained by a discriminator, wherein the output of the PointNet + + is a probability value smaller than 1;

when the output of PointNet + + is close to 1, the current input point cloud data is point cloud data waiting for reconstruction, and when the output of PointNet + + is close to 0, the current input point cloud data generated from the generator is represented.

5. The method for reconstructing a point cloud based on a generated countermeasure network as claimed in claim 1, wherein the training optimization is performed on the discriminator and the generator for a plurality of times to obtain a trained discriminator and a trained generator, and specifically:

optimizing the network by adopting a loss function of the least square GANs, and generating a loss function of the countermeasure network, wherein the loss function of the countermeasure network comprises a loss function of the discriminator and a loss function of the generator;

and (3) calculating a loss function of the discriminator according to the formula:

x represents the point cloud data to be reconstructed, D (X) represents the probability value obtained by inputting X into the discriminator, G (X) represents the point cloud data generated from the generator, D (G (X)) represents the probability value obtained by inputting G (X) into the discriminator, and L (X) represents the probability value obtained by inputting X into the discriminator_DThe final discriminator loss;

the loss function of the generator is calculated as shown in the formula:

L_G＝γ₁L_GD+γ₂D

wherein gamma is₁And gamma₂Is a self-defined parameter which can be adjusted according to the actual situation, L_GDTo combat the loss, calculating the error of the discriminator for discriminating the point cloud data generated by the generator as real point cloud data, wherein X represents the whole point cloud data set waiting for reconstruction, X represents the point cloud data waiting for reconstruction, G (X) represents the point cloud data generated by the generator, and D (G (X)) represents the probability value obtained by inputting G (X) into the discriminator;

d is a bidirectional chamfering distance which refers to the bidirectional chamfering distance between the point cloud data waiting for reconstruction and the point cloud data generated by the generator, wherein X and Y respectively represent the point cloud data waiting for reconstruction and the point cloud data generated by the generator, X and Y respectively represent the coordinate of one point in the point cloud data waiting for reconstruction and the point cloud data generated by the generator, and L is_GThe final generator loss;

the actual training times are determined according to the complexity of the required three-dimensional model, and training optimization is carried out for multiple times to obtain a trained discriminator and a trained generator.

6. A system for point cloud three-dimensional reconstruction based on generation of countermeasure network, the system comprising:

the data processing unit is used for processing the point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed;

the module building unit is used for building a generator module and a discriminator module;

and the module training unit is used for carrying out multiple times of training optimization on the discriminator and the generator to obtain the trained discriminator and the trained generator, and the three-dimensional model corresponding to the point cloud data to be reconstructed is generated by the finally trained generator.

7. The system of claim 6, wherein the data processing unit normalizes the point cloud data to be reconstructed to normalize the coordinates of each point to between-1 and 1, if the three-dimensional model to be obtained is formed by splicing different parts of a plurality of three-dimensional models, the point cloud of the part to be spliced needs to be put at a corresponding position, and the normalization is also performed, and the point cloud data obtained after the normalization is processed by using three-dimensional modeling software to obtain a three-dimensional mesh model capable of surrounding the point cloud data.

8. The system of claim 6, wherein the module building unit builds a generator module, the generator module adopts MeshCNN, the input is a three-dimensional grid model, the displacement of each vertex in the model is obtained, the displacement is added to the vertex coordinates of the three-dimensional grid model to obtain a new three-dimensional grid model, points are uniformly sampled from the new model to obtain the point cloud data generated by the generator, the discriminator module is built, the input is the point cloud data to be reconstructed and the point cloud data generated from the generator, and the output is a probability value smaller than 1.

9. The system of claim 6, wherein the module training unit optimizes the network by using a loss function of least Squares (GANs), the loss function of the generated countermeasure network comprises a loss function of the discriminator and a loss function of the generator, the actual training times are determined according to the complexity of the required three-dimensional model, the trained discriminator and the trained generator are obtained by training and optimizing for multiple times, and the three-dimensional model corresponding to the point cloud data to be reconstructed is generated by the finally trained generator.

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 5.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.

Technical Field

The invention relates to the field of three-dimensional reconstruction and deep learning, in particular to a point cloud three-dimensional reconstruction method, a system, equipment and a storage medium based on a generated countermeasure network.

Background

In recent years, with the continuous development of internet and multimedia technology and the popularization of digital equipment and mass storage equipment, the application of three-dimensional point cloud data is in the field of vision of people. For example, in virtual reality, people can obtain a three-dimensional model through hand drawing. For example, in automatic driving, the three-dimensional point cloud data scanned by laser is processed to make the computer able to identify the environment where the vehicle is located at that time according to the point cloud data, and know where there are obstacles, so as to deduce how the vehicle will pass after the vehicle is driven. In addition, three-dimensional models are widely used in video and game production.

Three-dimensional reconstruction techniques are an important direction of research in computer graphics. It refers to the process of processing some known data to infer the original three-dimensional model. For example, an original three-dimensional object is synthesized by a plurality of pictures of an object from different angles, or a three-dimensional object is estimated by an image with depth information, or a three-dimensional model is obtained by three-dimensional point cloud data. With the rapid development of deep learning in recent years, people also apply the deep learning to the task of classifying and segmenting three-dimensional point cloud data, and a better effect is achieved. The point clouds can be subjected to semantic segmentation better through a computer in a deep learning technology, so that the point clouds belonging to the same class can be known, and the method has important significance in the research of the automatic driving technology. In addition, deep learning is also applied to the research of three-dimensional reconstruction, and a better effect is achieved.

One of the prior art at present is a method, a system, a medium and a device for three-dimensional reconstruction of a pipeline based on deep learning, and the technical scheme is as follows: (1) learning the characteristics of the point cloud by using a deep learning method to at least obtain the category of the component to which the point belongs, the radius of the component to which the point belongs and the direction vector of the point; (2) calculating an axis point by using the radius of the component to which the point belongs and the direction vector of the point, and clustering the axis point by combining the category label of the component to which the point belongs to obtain a candidate example; (3) obtaining connection relations among different candidate examples by a graph-based method, and forming a graph structure by taking assemblies as nodes; (4) and replacing nodes in the graph with the actual three-dimensional component model to complete the reconstruction of the whole pipeline. The disadvantages are that: the scheme is not used for performing three-dimensional reconstruction on point cloud data, but the existing three-dimensional components are used for completing the whole pipeline through the components of the point clouds to the points, and the scheme is specially used for achieving three-dimensional reconstruction of the pipeline and is not strong in universality.

Another prior art is a point cloud three-dimensional reconstruction method based on RGB data and generation of a countermeasure network, the technical scheme being: (1) capturing a two-dimensional image using an inexpensive general camera; (2) generating a depth image estimate from the single RGB image by generating a countermeasure network; (3) generating a depth image estimation by generating parameters of a training data depth camera, and calculating according to the depth to obtain three-dimensional point cloud data; (4) and obtaining a rough surface model through spherical mapping, and finally judging the model by using a discriminator to obtain a complete three-dimensional model. The disadvantages are that: (1) according to the scheme, point cloud data are obtained through a two-dimensional image, and then three-dimensional reconstruction is carried out. Note that point cloud data is obtained from a two-dimensional image, which is achieved by estimation, and therefore a certain loss must occur here. (2) In addition, simple spherical surface mapping is adopted in the three-dimensional reconstruction process, and a complete three-dimensional model is obtained through refinement, the three-dimensional model obtained through the method is only approximate roughly, and the obtained three-dimensional model is not accurate enough.

Disclosure of Invention

The invention aims to overcome the defects of the existing method and provides a point cloud three-dimensional reconstruction method based on a generation countermeasure network. The invention solves the main problems that: the method has the advantages that the existing three-dimensional reconstruction technology is designed for special objects or scenes mostly, and universality is not strong enough; secondly, the problem of data loss caused by point cloud data acquired through a two-dimensional image; thirdly, the three-dimensional model established through simple mapping causes the problem of low model precision.

In order to solve the above problems, the present invention provides a method for point cloud three-dimensional reconstruction based on generation of a countermeasure network, the method comprising:

processing point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed;

building a generator module, inputting the three-dimensional grid model, obtaining the displacement of each vertex in the model, adding the displacement and the vertex coordinates of the three-dimensional grid model to obtain a new three-dimensional grid model, and uniformly sampling some points from the new model to obtain point cloud data generated by the generator;

constructing a discriminator module, inputting the point cloud data waiting for reconstruction and the point cloud data generated from the generator, and outputting a probability value smaller than 1;

training and optimizing the discriminator and the generator for multiple times to obtain a trained discriminator and a trained generator;

and finally, the generator after training generates a three-dimensional model corresponding to the point cloud data to be reconstructed.

Preferably, the processing of the point cloud data to be reconstructed to obtain a three-dimensional mesh model surrounding the point cloud data to be reconstructed specifically includes:

carrying out normalization processing on point cloud data to be reconstructed, and normalizing the coordinates of each point to be between-1 and 1;

if the three-dimensional model to be obtained is formed by splicing different parts of a plurality of three-dimensional models, point clouds of the parts to be spliced are required to be placed at corresponding positions, and normalization processing is also carried out;

and (4) using three-dimensional modeling software to obtain a three-dimensional grid model capable of surrounding the point cloud data after the normalization treatment.

Preferably, the generator module is set up, the three-dimensional mesh model is input, the displacement of each vertex in the model is obtained, the displacement is added to the vertex coordinates of the three-dimensional mesh model to obtain a new three-dimensional mesh model, some points are uniformly sampled from the new model to obtain point cloud data generated by the generator, and the generator module specifically comprises:

the generator adopts MeshCNN, is a neural network acting on each edge in the three-dimensional mesh, and firstly calculates five characteristics possessed by each edge according to the three-dimensional mesh, wherein the five characteristics are dihedral angles between two triangular patches where each edge is located, and the ratios of the height of opposite angles in the two triangular patches where the edge is located and the height of the two triangular patches where the edge is located to the edge;

the calculated characteristic is similar to the convolution neural network of the image and is sent to the network for calculation, so that the displacement required by each vertex in the three-dimensional grid is obtained;

adding the displacement of the point and the vertex coordinates of the three-dimensional mesh model to obtain a three-dimensional mesh model closer to real point cloud data;

and uniformly sampling some points from the new model to obtain point cloud data generated by the generator.

Preferably, the set-up discriminator module inputs the point cloud data to be reconstructed and the point cloud data generated from the generator, and outputs a probability value smaller than 1, specifically:

the discriminator is a two-classifier, a PointNet + + network model is adopted as the discriminator, and the point cloud data waiting for reconstruction and the point cloud data generated from the generator are input;

modifying the output number of the PointNet + + full connection to be 1, judging whether the point cloud data waiting for reconstruction is obtained by a discriminator, wherein the output of the PointNet + + is a probability value smaller than 1;

when the PointNet + + output is close to 1, it indicates that the currently input point cloud data is actually the point cloud data, and when the PointNet + + output is close to 0, it indicates that the currently input is the point cloud data generated from the generator.

Preferably, the training optimization is performed on the discriminator and the generator for multiple times to obtain a trained discriminator and a trained generator, and specifically includes:

optimizing the network by adopting a loss function of the least square GANs, and generating a loss function of the countermeasure network, wherein the loss function of the countermeasure network comprises a loss function of the discriminator and a loss function of the generator;

and (3) calculating a loss function of the discriminator according to the formula:

x represents the point cloud data to be reconstructed, D (X) represents the probability value obtained by inputting X into the discriminator, G (X) represents the point cloud data generated from the generator, D (G (X)) represents the probability value obtained by inputting G (X) into the discriminator, and L (X) represents the probability value obtained by inputting X into the discriminator_DThe final discriminator loss;

the loss function of the generator is calculated as shown in the formula:

L_G＝γ₁L_GD+γ₂D

wherein gamma is₁And gamma₂Is a self-defined parameter which can be adjusted according to the actual situation, L_GDTo combat the loss, calculating the error of the discriminator for discriminating the point cloud data generated by the generator as real point cloud data, wherein X represents the whole point cloud data set waiting for reconstruction, X represents the point cloud data waiting for reconstruction, G (X) represents the point cloud data generated by the generator, and D (G (X)) represents the probability value obtained by inputting G (X) into the discriminator;

d is a bidirectional chamfering distance which refers to the bidirectional chamfering distance between the point cloud data waiting for reconstruction and the point cloud data generated by the generator, wherein X and Y respectively represent the point cloud data waiting for reconstruction and the point cloud data generated by the generator, X and Y respectively represent the coordinate of one point in the point cloud data waiting for reconstruction and the point cloud data generated by the generator, and L is_GThe final generator loss;

the actual training times are determined according to the complexity of the required three-dimensional model, and training optimization is carried out for multiple times to obtain a trained discriminator and a trained generator.

Correspondingly, the invention also provides a system for point cloud three-dimensional reconstruction based on the generation countermeasure network, which comprises:

the data processing unit is used for processing the point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed;

the module building unit is used for building a generator module and a discriminator module;

and the module training unit is used for carrying out multiple times of training optimization on the discriminator and the generator to obtain the trained discriminator and the trained generator, and the three-dimensional model corresponding to the point cloud data to be reconstructed is generated by the finally trained generator.

Correspondingly, the invention also provides computer equipment which comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the point cloud three-dimensional reconstruction method based on the generation countermeasure network.

Accordingly, the present invention also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the above-mentioned steps of a method for three-dimensional reconstruction of a point cloud based on generation of a countermeasure network.

The implementation of the invention has the following beneficial effects:

firstly, most of the point cloud data obtained by laser scanning in real life is obtained by laser scanning, however, most of the point cloud data obtained by laser scanning in real life are not uniformly distributed, some very sparse places can be generated, and the situations of normal vector deficiency or errors can occur, and the undesirable situations have great influence on three-dimensional reconstruction, but the invention can better cope with the situations; secondly, smooth transition areas can be generated when the task of splicing different parts of a plurality of different three-dimensional models is processed, the transition areas can not be simply spliced together, and in addition, because the grid compositions of the three-dimensional models are different, triangular grids, quadrilateral grids and polygonal grids are possible, different grid models can be spliced by the scheme, and the triangular grids are finally obtained; thirdly, the scheme can be applied to various single objects, and the application in three-dimensional reconstruction is wide.

Drawings

FIG. 1 is a general flowchart of a point cloud three-dimensional reconstruction method based on generation of a countermeasure network according to an embodiment of the present invention;

FIG. 2 is a diagram of a generator computation feature according to an embodiment of the present invention;

fig. 3 is a structural diagram of a point cloud three-dimensional reconstruction system based on a generation countermeasure network according to an embodiment of the present invention.

Detailed Description

Technical inventions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Fig. 1 is a general flowchart of a point cloud three-dimensional reconstruction method based on generation of a countermeasure network according to an embodiment of the present invention, as shown in fig. 1, the method includes:

s1, processing the point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed;

s2, building a generator module, inputting the three-dimensional mesh model, obtaining the displacement of each vertex in the model, adding the displacement and the vertex coordinates of the three-dimensional mesh model to obtain a new three-dimensional mesh model, and uniformly sampling some points from the new model to obtain point cloud data generated by the generator;

s3, constructing a discriminator module, inputting the point cloud data waiting for reconstruction and the point cloud data generated from the generator, and outputting a probability value smaller than 1;

s4, performing training optimization on the discriminator and the generator for multiple times to obtain a trained discriminator and a trained generator;

and S5, finally, the generator after training generates the three-dimensional model corresponding to the point cloud data waiting for reconstruction.

Step S1 is specifically as follows:

s1-1: and (4) carrying out normalization processing on the point cloud data to be reconstructed, and normalizing the coordinates of each point to be between-1 and 1. If the three-dimensional model to be obtained is formed by splicing different parts of a plurality of three-dimensional models, the point clouds of the parts to be spliced are required to be placed at corresponding positions, and normalization processing is also required.

S1-2: using three-dimensional modeling software such as Blend or the like, a triangular mesh model capable of surrounding the point cloud data is obtained from the point cloud data obtained in S1-1.

Step S2 is specifically as follows:

s2-1: as shown in fig. 2, the generator employs a mescnn, which is a neural network acting on each edge of the three-dimensional mesh, whose input is a closed three-dimensional mesh model that can enclose the original point cloud data. It first calculates the five characteristics of each edge according to the three-dimensional grid. The five characteristics are the dihedral angle between the two triangular patches where each edge is located, the opposite angle of the two triangular patches where the edge is located and the ratio of the height of the two triangular patches where the edge is located to the edge.

S2-2: and (4) sending the convolutional neural network with the characteristics which are calculated in the S2-1 and similar to the image to the network for calculation, thereby obtaining the displacement which is required by each vertex in the three-dimensional mesh. And then adding the displacement of the point and the coordinates of the vertex of the original three-dimensional mesh to obtain a three-dimensional mesh which is closer to the real point cloud data. And uniformly sampling some points from the new three-dimensional mesh model so as to obtain point cloud data generated by the generator.

Step S3 is specifically as follows:

s3-1: the discriminator is a two-classifier overall, a PointNet + + network model with better performance in point cloud data classification is used as the discriminator, and the input is the point cloud data waiting for reconstruction and the point cloud data obtained by sampling from the three-dimensional model generated in the generator.

S3-2: the output number after the PointNet + + full connection is modified is 1, and because the two classifications are carried out here, only the point cloud data waiting for reconstruction need to be judged. The output of PointNet + + is a probability value less than 1. When the output of PointNet + + is close to 1, the current input point cloud data is actually the point cloud data, and when the output of PointNet + + is close to 0, the current input point cloud data sampled from the three-dimensional model generated by the generator is represented.

Step S4 is specifically as follows:

s4-1: because the generation of the confrontation network is difficult to train, the scheme refers to least square GANs and adopts the loss function thereof to optimize the network model. Generating the loss function for the counterpoise network includes the loss function of the arbiter and the loss function of the generator.

S4-2: the formula for calculating the loss function of the discriminator is shown in formula (1). Wherein X represents the point cloud data to be reconstructed, D (X) represents the probability value obtained by inputting X into the discriminator, G (X) represents the point cloud data obtained from the generator, D (G (X)) represents the probability value obtained by inputting G (X) into the discriminator, L (X) represents the probability value obtained by inputting G (X) into the discriminator_DIt is the final discriminator loss for the feedback propagation optimization discrimination network.

S4-3: the generator loss function calculation formula is shown in formula (4). Wherein gamma is₁And gamma₂Is a self-defined parameter, is suggested to be set to 0.01 and 100, can be adjusted according to actual conditions, and L_GDThe calculation of D is shown in equations (2) and (3). Formula (2) calculates the error of the discriminator for discriminating the point cloud data generated by the generator into real point cloud data, wherein X represents the whole point cloud data set waiting for reconstruction, g (X) represents the point cloud data set obtained by inputting X into the generator, and D (g (X)) represents the probability value obtained by inputting g (X) into the discriminator. Formula (3) meterCalculating the bidirectional chamfering distance between the point cloud data waiting for reconstruction and the generated point cloud data, wherein X and Y respectively represent the point cloud data waiting for reconstruction and the generated point cloud data, and X and Y respectively represent the coordinate of one point in the point cloud data waiting for reconstruction and the generated point cloud data.

L_G＝γ₁L_GD+γ₂D (4)

Correspondingly, the invention also provides a system for point cloud three-dimensional reconstruction based on generation of a countermeasure network, as shown in fig. 3, comprising:

the data processing unit 1 processes the point cloud data to be reconstructed to obtain a three-dimensional grid model surrounding the point cloud data to be reconstructed.

Specifically, point cloud data to be reconstructed is normalized, coordinates of each point are normalized to be between-1 and 1, if a three-dimensional model to be obtained is formed by splicing different parts of a plurality of three-dimensional models, the point cloud of the part to be spliced is required to be placed at a corresponding position, the normalization processing is also performed, and the point cloud data obtained after the normalization processing is processed by using three-dimensional modeling software such as Blend and the like, so that a three-dimensional mesh model capable of surrounding the point cloud data is obtained.

And the module building unit 2 builds a generator module and a discriminator module.

Specifically, a generator module is built, a MeshCNN is adopted by the generator, a three-dimensional grid model is input, the displacement of each vertex in the model is obtained, the displacement is added with the vertex coordinates of the three-dimensional grid model to obtain a new three-dimensional grid model, some points are uniformly sampled from the new model to obtain point cloud data generated by the generator, a discriminator module is built, the point cloud data to be reconstructed and the point cloud data generated from the generator are input, and the output probability value is smaller than 1.

And the module training unit 3 is used for carrying out multiple times of training optimization on the discriminator and the generator to obtain the trained discriminator and the trained generator, and the three-dimensional model corresponding to the point cloud data to be reconstructed is generated by the finally trained generator.

Specifically, a loss function of least squares GANs is adopted to optimize the network, the generated loss function of the countermeasure network comprises a loss function of a discriminator and a loss function of a generator, the actual training times are determined according to the complexity of the required three-dimensional model, the training optimization is carried out for multiple times, the trained discriminator and the trained generator are obtained, and the three-dimensional model corresponding to the point cloud data waiting for reconstruction is generated by the finally trained generator.

Therefore, the invention constructs and generates the countermeasure network by constructing the generator module and the discriminator module and using PointNet + + as the discriminator to carry out the three-dimensional reconstruction of the point cloud; meanwhile, the loss function of the discriminator refers to LSGANs (least squares GANs), the loss function of the generator resists loss and linear combination of the bidirectional chamfering distance between the real point cloud data and the point cloud data obtained by sampling, the secondary loss function can better reflect the three-dimensional characteristic of the point cloud data, the convergence speed of the network model can be accelerated, and the result obtained by the generator can be better converged to a place expected by people; finally, the corresponding three-dimensional model is finally obtained by taking the owned point cloud data as a reference, the method can be applied to various single objects, and smooth transition areas can be generated in splicing tasks of different parts of a plurality of models without very abrupt changes.

Correspondingly, the invention also provides computer equipment which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the point cloud three-dimensional reconstruction method based on the generation countermeasure network when executing the computer program. Meanwhile, the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the above-mentioned point cloud three-dimensional reconstruction method based on the generation countermeasure network.

The method, the system, the equipment and the storage medium for point cloud three-dimensional reconstruction based on generation of the countermeasure network provided by the embodiment of the invention are described in detail, a specific example is applied in the description to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.