阅读说明：本技术 点云分割方法、装置及终端设备 (Point cloud segmentation method and device and terminal equipment ) 是由 陈方平 唐宇鹏 陆煜衡 于 2021-08-24 设计创作，主要内容包括：本发明提供了一种点云分割方法、装置及终端设备,包括：如果接收到点云分割请求,调用预设接口读取待分割点云数据；基于待分割点云数据的点数量,判断待分割点云数据是否满足第一预设条件；如果否,对待分割点云数据进行切块处理得到多个点云数据块,直至每个点云数据块均满足第一预设条件；对每个点云数据块分别进行降采样处理得到目标点云数据；将目标点云数据发送至服务器,以使服务器利用点云分割网络对目标点云数据进行分割处理得到分割结果；接收服务器反馈的分割结果。本发明可以有效减少点云分割任务对使用人员经验的依赖,还可以显著提高点云分割的准确性。(The invention provides a point cloud segmentation method, a point cloud segmentation device and terminal equipment, wherein the point cloud segmentation device comprises the following steps: if a point cloud segmentation request is received, calling a preset interface to read point cloud data to be segmented; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets a first preset condition; respectively performing down-sampling processing on each point cloud data block to obtain target point cloud data; sending the target point cloud data to a server so that the server performs segmentation processing on the target point cloud data by using a point cloud segmentation network to obtain a segmentation result; and receiving the segmentation result fed back by the server. The invention can effectively reduce the dependence of the point cloud segmentation task on the experience of users and can also obviously improve the accuracy of point cloud segmentation.)

1. A point cloud segmentation method is applied to a client which is in communication connection with a server, and the server is configured with a point cloud segmentation network, and the method comprises the following steps:

if a point cloud segmentation request is received, calling a preset interface to read point cloud data to be segmented;

judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented;

if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition;

performing down-sampling processing on each point cloud data block to obtain target point cloud data;

sending the target point cloud data to the server so that the server performs segmentation processing on the target point cloud data by using the point cloud segmentation network to obtain a segmentation result;

and receiving the segmentation result fed back by the server.

2. The method according to claim 1, wherein the step of performing a dicing process on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition includes:

acquiring preset cutting parameters, and cutting the point cloud data to be segmented based on the cutting parameters to obtain a plurality of point cloud data blocks;

judging whether each point cloud data block meets the first preset condition or not;

and if the point cloud data blocks do not meet the first preset condition, carrying out block cutting processing on the point cloud data blocks based on the block cutting parameters until each point cloud data block meets the first preset condition.

3. The method of claim 1, wherein the down-sampling each point cloud data block to obtain target point cloud data comprises:

acquiring a preset down-sampling parameter; wherein the down-sampling parameter comprises a down-sampling radius;

for each point cloud data block, judging whether the point cloud data block meets a second preset condition or not based on the point density of the point cloud data block;

if not, performing at least one time of downsampling processing on the point cloud data block until the point cloud data block meets the second preset condition;

and determining each point cloud data block meeting the second preset condition as target point cloud data.

4. The method of claim 3, wherein the step of down-sampling the point cloud data block at least once comprises:

for one-time downsampling processing, a target point is randomly determined from the point cloud data block, a search area is determined by taking the target point as a circle center and the downsampling radius as a circle radius, and a specified number of data points are randomly determined from the data points contained in the search area.

5. A point cloud segmentation method is applied to a server which is in communication connection with a client, and the server is configured with a point cloud segmentation network, and the method comprises the following steps:

receiving target point cloud data sent by the client; the target point cloud data is obtained by the client based on the point cloud data to be segmented, and the client is used for calling a preset interface to read the point cloud data to be segmented when receiving a point cloud segmentation request; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition; performing down-sampling processing on each point cloud data block to obtain target point cloud data;

carrying out segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data; the point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data;

and sending the segmentation result to the client.

6. The method according to claim 5, wherein the step of performing segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data comprises:

converting the target point cloud data into voxel data through the feature aggregation sub-network, and performing feature aggregation processing on the voxel data to obtain structural features of the target point cloud data;

performing feature extraction processing on each point in the target point cloud data through the feature extraction sub-network to obtain detailed features of the target point cloud data;

and carrying out feature fusion processing on the structural features and the detail features to obtain a segmentation result corresponding to the target point cloud data.

7. A point cloud segmentation apparatus, wherein the apparatus is applied to a client, the client is in communication connection with a server, the server is configured with a point cloud segmentation network, and the apparatus comprises:

the data reading module is used for calling a preset interface to read point cloud data to be segmented if a point cloud segmentation request is received;

the judging module is used for judging whether the point cloud data to be segmented meets a first preset condition or not based on the number of points of the point cloud data to be segmented;

the block cutting module is used for cutting the cloud point data to be segmented to obtain a plurality of point cloud data blocks when the judgment result of the judgment module is yes, until each point cloud data block meets the first preset condition;

the down-sampling module is used for respectively performing down-sampling processing on each point cloud data block to obtain target point cloud data;

the data sending module is used for sending the target point cloud data to the server so that the server can carry out segmentation processing on the target point cloud data by utilizing the point cloud segmentation network to obtain a segmentation result;

and the result receiving module is used for receiving the segmentation result fed back by the server.

8. A point cloud segmentation apparatus applied to a server, the server being in communication connection with a client, the server being configured with a point cloud segmentation network, the apparatus comprising:

the data receiving module is used for receiving the target point cloud data sent by the client; the target point cloud data is obtained by the client based on the point cloud data to be segmented, and the client is used for calling a preset interface to read the point cloud data to be segmented when receiving a point cloud segmentation request; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition; performing down-sampling processing on each point cloud data block to obtain target point cloud data;

the segmentation module is used for carrying out segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data; the point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data;

and the result sending module is used for sending the segmentation result to the client.

9. A terminal device comprising a processor and a memory, the memory storing computer-executable instructions executable by the processor to perform the method of any one of claims 1 to 4 or to perform the method of any one of claims 5 to 6.

10. A computer-readable storage medium having stored thereon computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any of claims 1 to 4, or cause the processor to implement the method of any of claims 5 to 6.

Technical Field

The invention relates to the technical field of point cloud segmentation, in particular to a point cloud segmentation method, a point cloud segmentation device and terminal equipment.

Background

The point cloud segmentation technology is widely applied to the technical fields of surveying and mapping, automatic driving, agriculture, design planning and the like. At present, application software for point cloud segmentation has high requirements on users, the users need to have high experience when performing point cloud segmentation on complex scenes, and the requirements on labor cost are also high. The point cloud segmentation software provided by the related technology has higher experience dependence on users and has the problem of lower point cloud segmentation accuracy.

Disclosure of Invention

In view of this, the present invention provides a point cloud segmentation method, a point cloud segmentation device, and a terminal device, which can effectively reduce the dependence of a point cloud segmentation task on the experience of a user, and can significantly improve the accuracy of point cloud segmentation.

In a first aspect, an embodiment of the present invention provides a point cloud segmentation method, where the method is applied to a client, the client is in communication connection with a server, and the server is configured with a point cloud segmentation network, and the method includes: if a point cloud segmentation request is received, calling a preset interface to read point cloud data to be segmented; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition; performing down-sampling processing on each point cloud data block to obtain target point cloud data; sending the target point cloud data to the server so that the server performs segmentation processing on the target point cloud data by using the point cloud segmentation network to obtain a segmentation result; and receiving the segmentation result fed back by the server.

In one embodiment, the step of performing a block cutting process on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition includes: acquiring preset cutting parameters, and cutting the point cloud data to be segmented based on the cutting parameters to obtain a plurality of point cloud data blocks; judging whether each point cloud data block meets the first preset condition or not; and if the point cloud data blocks do not meet the first preset condition, carrying out block cutting processing on the point cloud data blocks based on the block cutting parameters until each point cloud data block meets the first preset condition.

In one embodiment, the down-sampling each point cloud data block to obtain target point cloud data includes: acquiring a preset down-sampling parameter; wherein the down-sampling parameter comprises a down-sampling radius; for each point cloud data block, judging whether the point cloud data block meets a second preset condition or not based on the point density of the point cloud data block; if not, performing at least one time of downsampling processing on the point cloud data block until the point cloud data block meets the second preset condition; and determining each point cloud data block meeting the second preset condition as target point cloud data.

In one embodiment, the step of performing at least one down-sampling process on the point cloud data block includes: for one-time downsampling processing, a target point is randomly determined from the point cloud data block, a search area is determined by taking the target point as a circle center and the downsampling radius as a circle radius, and a specified number of data points are randomly determined from the data points contained in the search area.

In a second aspect, an embodiment of the present invention further provides a point cloud segmentation method, where the method is applied to a server, the server is in communication connection with a client, and the server is configured with a point cloud segmentation network, and the method includes: receiving target point cloud data sent by the client; the target point cloud data is obtained by the client based on the point cloud data to be segmented, and the client is used for calling a preset interface to read the point cloud data to be segmented when receiving a point cloud segmentation request; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition; performing down-sampling processing on each point cloud data block to obtain target point cloud data; carrying out segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data; the point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data; and sending the segmentation result to the client.

In an embodiment, the step of performing segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data includes: converting the target point cloud data into voxel data through the feature aggregation sub-network, and performing feature aggregation processing on the voxel data to obtain structural features of the target point cloud data; performing feature extraction processing on each point in the target point cloud data through the feature extraction sub-network to obtain detailed features of the target point cloud data; and carrying out feature fusion processing on the structural features and the detail features to obtain a segmentation result corresponding to the target point cloud data.

In one embodiment, the step of converting the target point cloud data into voxel data and performing feature aggregation processing on the voxel data to obtain the structural features of the target point cloud data includes: carrying out normalization processing on the target point cloud data to obtain normalized data; converting the normalized data into voxel data according to a preset conversion formula; performing feature aggregation processing on the voxel data to obtain a voxel feature corresponding to the voxel data; and performing inverse voxel processing on the voxel characteristics to obtain the structural characteristics of the target point cloud data.

In one embodiment, the feature extraction sub-network comprises a multi-layer perceptron; the step of performing feature extraction processing on each point in the target point cloud data through the feature extraction sub-network to obtain the detailed features of the target point cloud data includes: and performing feature extraction processing on each point in the target point cloud data through the multilayer perceptron to obtain the detail features of the target point cloud data.

In a third aspect, an embodiment of the present invention further provides a point cloud segmentation apparatus, where the apparatus is applied to a client, the client is in communication connection with a server, and the server is configured with a point cloud segmentation network, and the apparatus includes: the data reading module is used for calling a preset interface to read point cloud data to be segmented if a point cloud segmentation request is received; the judging module is used for judging whether the point cloud data to be segmented meets a first preset condition or not based on the number of points of the point cloud data to be segmented; the block cutting module is used for cutting the point cloud data to be segmented to obtain a plurality of point cloud data blocks when the judgment result of the judgment module is yes, until each point cloud data block meets the first preset condition; the down-sampling module is used for respectively performing down-sampling processing on each point cloud data block to obtain target point cloud data; the data sending module is used for sending the target point cloud data to the server so that the server can carry out segmentation processing on the target point cloud data by utilizing the point cloud segmentation network to obtain a segmentation result; and the result receiving module is used for receiving the segmentation result fed back by the server.

In a fourth aspect, an embodiment of the present invention further provides a point cloud segmentation apparatus, where the apparatus is applied to a server, the server is in communication connection with a client, and the server is configured with a point cloud segmentation network, and the apparatus includes: the data receiving module is used for receiving the target point cloud data sent by the client; the target point cloud data is obtained by the client based on the point cloud data to be segmented, and the client is used for calling a preset interface to read the point cloud data to be segmented when receiving a point cloud segmentation request; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting processing on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets the first preset condition; performing down-sampling processing on each point cloud data block to obtain target point cloud data; the segmentation module is used for carrying out segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data; the point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data; and the result sending module is used for sending the segmentation result to the client.

In a fifth aspect, an embodiment of the present invention further provides a terminal device, including a processor and a memory, where the memory stores computer-executable instructions capable of being executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect, or executes the computer-executable instructions to implement any one of the methods provided in the second aspect.

In a sixth aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any one of the first aspect, or cause the processor to implement the method of any one of the second aspect.

The embodiment of the invention provides a point cloud segmentation method, a point cloud segmentation device and terminal equipment, which are applied to a client, wherein the client is in communication connection with a server, the server is provided with a point cloud segmentation network, calling a preset interface to read point cloud data to be segmented when a point cloud segmentation request is received, judging whether the point cloud data to be segmented meets a first preset condition or not based on the number of points of the point cloud data to be segmented, if the point cloud data to be segmented does not meet the first preset condition, then the point cloud data to be segmented is cut into a plurality of point cloud data blocks until each point cloud data block meets a first preset condition, then, each point cloud data block is respectively subjected to down-sampling processing to obtain target point cloud data, the target point cloud data is sent to a server, the server uses the point cloud segmentation network to segment the target point cloud data to obtain segmentation results, and receives the segmentation results fed back by the server. According to the method, after a point cloud segmentation request is received, the operations of reading point cloud data to be segmented, blocking processing, downsampling processing, data sending, result receiving and the like can be automatically executed, the automation level of the segmented point cloud data is obviously improved, a point cloud segmentation network configured in a server is used for segmenting target point cloud data, a final segmentation result is obtained based on structural features and detail features, the accuracy of point cloud segmentation can be obviously improved, a user does not need to adjust the segmentation result again, and the dependence of a point cloud segmentation task on the experience of the user can be effectively reduced.

The point cloud segmentation method, the point cloud segmentation device and the terminal equipment are applied to a server, the server is in communication connection with a client, the server is configured with a point cloud segmentation network, firstly, target point cloud data sent by the client are received, then, the target point cloud data are segmented through the point cloud segmentation network, segmentation results corresponding to the target point cloud data are obtained, and finally, the segmentation results are sent to the client. The point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data. The method can remarkably improve the automation level of the point cloud data to be segmented by segmenting the target point cloud data obtained by the client based on the point cloud data to be segmented, and respectively extracts the structural features and the detail features of the target point cloud data by utilizing the point cloud segmentation network comprising the feature aggregation sub-network and the feature extraction sub-network, thereby obtaining the segmentation result on the basis of the structural features and the detail features and remarkably improving the accuracy of point cloud segmentation.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic flow chart of a point cloud segmentation method according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of another point cloud segmentation method according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a point cloud segmentation network according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of another point cloud segmentation method according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a point cloud segmentation apparatus according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of another point cloud segmentation apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the embodiments, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

At present, the traditional point cloud segmentation software depends on the experience of users, special points are arranged in the software, and then point clouds are classified according to distance, angles and the like, so that the method has a poor segmentation effect on fine point clouds, and adjustment of professional marking personnel is needed after segmentation.

To facilitate understanding of the present embodiment, first, a point cloud segmentation method disclosed in the present embodiment is described in detail, where the method is applied to a client, the client is in communication connection with a server, and the server is configured with a point cloud segmentation network, and referring to a flowchart of the point cloud segmentation method shown in fig. 1, the method mainly includes the following steps S102 to S112:

and step S102, if a point cloud segmentation request is received, calling a preset interface to read point cloud data to be segmented. In one embodiment, a client may provide a graphical user interface, the graphical user interface displays a "point cloud segmentation" control, when a user clicks the "point cloud segmentation" control, a point cloud segmentation request may be generated, in addition, the client is configured with a storage area, a preset interface is configured with a website (socket) and a reading path of the storage area, the user only needs to store the point cloud to be segmented into the storage area, and after the client receives the point cloud segmentation request sent by the user, the preset interface may be called to automatically read the point cloud data to be segmented from the storage area according to the reading path.

And step S104, judging whether the point cloud data to be segmented meets a first preset condition or not based on the number of points of the point cloud data to be segmented. The first preset condition can be understood as a quantity condition, and the running load of the server in the subsequent processing can be reduced by limiting the number of points contained in the point cloud to be segmented. In an implementation manner, a quantity threshold may be preset, and it is determined whether the quantity of the point cloud data to be segmented is smaller than the quantity threshold, if so, it may be determined that the point cloud data to be segmented satisfies a first preset condition, and directly skip to the downsampling step, and if not, the point cloud data to be segmented needs to be subjected to block processing, so that each point cloud data block obtained by the block processing satisfies the first preset condition.

And S106, if not, performing block cutting on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets a first preset condition. In an embodiment, a dicing parameter may be configured in advance, the dicing parameter may include the number of dicing blocks, for example, if the number of dicing blocks is 8, the point cloud data to be segmented is cut into 8 point cloud data blocks, at this time, it may be further determined whether each point cloud data block satisfies the first preset condition, and the dicing process is continued for the point cloud data blocks that do not satisfy the first preset condition until each point cloud data block satisfies the first preset condition.

And S108, respectively performing down-sampling processing on each point cloud data block to obtain target point cloud data. And after each point cloud data block meets the first preset condition, skipping to a down-sampling processing flow, and performing down-sampling processing on each point cloud data block respectively to reduce the point density of each point cloud data block, so that the operation burden of the server in subsequent processing is further reduced. In one embodiment, the down-sampling parameters may be configured in advance, so that each point cloud data block is down-sampled based on the down-sampling parameters, so that the down-sampled point cloud data block meets a second preset condition, where the second preset condition may be understood as a density condition. And when each point cloud data block meets the second preset condition, skipping to a data sending process, and sending each point cloud data block to a server as target point cloud data so that the server performs segmentation processing on the target point cloud data by using a point cloud segmentation network. The embodiment of the invention can automatically read the point cloud data to be segmented, automatically perform the block processing on the point cloud data to be segmented which does not meet the first preset condition, automatically jump to the down-sampling processing when the point cloud data block obtained by the block processing meets the first preset condition, automatically jump to the data sending processing when the target point cloud data obtained by the sampling processing meets the second preset condition, and automatically perform the operations of data reading, data block segmentation, data down-sampling, data sending and the like on the point cloud data to be segmented without manual operation of a user, thereby reducing the workload of the user.

Step S110, the target point cloud data is sent to a server so that the server can perform segmentation processing on the target point cloud data by using a point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data. The point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting structural features of the target point cloud data, the feature extraction sub-network is used for extracting detailed features of the target point cloud data, the structural features are used for describing the overall structure of the target point cloud data, and the detailed features are used for describing the features of each point in the target point cloud data. In an embodiment, the client may send a point cloud segmentation request to the server, where the point cloud segmentation request may adopt an HTTP (Hypertext Transfer Protocol) request, the point cloud segmentation request carries the target point cloud data, and after receiving the point cloud segmentation request, the server inputs the target point cloud data to a point cloud segmentation network, extracts a structural feature from the feature aggregation sub-network, extracts a detailed feature from the feature extraction sub-network, performs feature fusion processing on the structural feature and the detailed feature to obtain a segmentation result, and then the server feeds the segmentation result back to the client.

Step S112, receiving the segmentation result fed back by the server. In practical application, the client can receive the segmentation result fed back by the server, and further, the client can display the segmentation result to a user through a result display interface.

According to the point cloud segmentation method provided by the embodiment of the invention, after a point cloud segmentation request is received, the operations of reading point cloud data to be segmented, blocking processing, down-sampling processing, data sending, result receiving and the like can be automatically executed, the automation level of the segmented point cloud data is obviously improved, a point cloud segmentation network configured in a server is used for segmenting target point cloud data, a final segmentation result is obtained based on structural features and detail features, the accuracy of point cloud segmentation can be obviously improved, so that a user does not need to adjust the segmentation result again, and the dependence of a point cloud segmentation task on the experience of the user can be effectively reduced.

For the foregoing step S104, the embodiment of the present invention provides an implementation manner for determining whether the point cloud data to be segmented satisfies a first preset condition: a quantity threshold value can be configured in advance, if the quantity of the point cloud data to be segmented is smaller than the quantity threshold value, the point cloud data to be segmented can be considered to meet a first preset condition, and then the point cloud data to be segmented can be directly subjected to down-sampling processing; if the number of the point cloud data to be segmented is larger than or equal to the number threshold, the point cloud data to be segmented can be considered to not meet the first preset condition, and at the moment, the point cloud data to be segmented needs to be subjected to block cutting processing until the point cloud data blocks obtained through block cutting meet the first preset condition.

For the foregoing step S106, an embodiment of the present invention provides an implementation manner for performing a block cutting process on point cloud data to be segmented to obtain a plurality of point cloud data blocks, which refers to the following steps 1 to 3:

step 1, obtaining preset slicing parameters, and carrying out slicing processing on point cloud data to be segmented based on the slicing parameters to obtain a plurality of point cloud data blocks. The dicing parameters may include the number of dices and may also include the dicing step size. In an embodiment, assuming that the number of the cut blocks is 8 and the step size of the cut blocks is 100, the point cloud data to be segmented is cut into the cut blocks according to the cut block parameters, and 8 point cloud data blocks are obtained.

And 2, judging whether each point cloud data block meets a first preset condition. If yes, go to step S108; if not, step 3 is executed. Assuming that the obtained point cloud data blocks include data blocks 1 to 8, it is required to determine whether the number of points of each point cloud data block is less than a number threshold, and for the point cloud data blocks with the number of points less than the number threshold, it is determined that the point cloud data blocks satisfy a first preset condition, for example, the data blocks 1 to 4 do not satisfy the first preset condition, and the data blocks 5 to 8 satisfy the first preset condition. In an optional implementation manner, whether the point cloud data blocks satisfy the first preset condition may be sequentially determined according to the name order of the point cloud data blocks, or whether each point cloud data block satisfies the first preset condition may be simultaneously determined in a parallel processing manner.

And 3, if the point cloud data blocks do not meet the first preset condition, carrying out block cutting processing on the point cloud data blocks based on the block cutting parameters until each point cloud data block meets the first preset condition. With reference to the above example, assuming that the data blocks 1 to 4 do not satisfy the first preset condition, the data blocks 1 to 4 need to be diced again, taking the data block 1 as an example, the data block 1 is diced based on the set dicing parameters, so as to obtain 8 point cloud data blocks corresponding to the data block 1, and the above operation is repeated on the 8 point cloud data blocks obtained by the dicing processing this time until the number of points of each obtained point cloud data block is less than the number threshold.

When each point cloud data block meets a first preset condition, the step of downsampling processing can be skipped, and the embodiment of the invention provides an implementation mode for downsampling each point cloud data block to obtain target point cloud data, which is shown in the following steps a to d:

step a, obtaining a preset down-sampling parameter. Wherein the down-sampling parameter comprises a down-sampling radius.

B, judging whether each point cloud data block meets a second preset condition or not based on the point density of the point cloud data block; if yes, go to step S110; if not, executing step c. Wherein the second preset condition may be a density condition. In an embodiment, a density threshold may be preset, if the point density of the point cloud data block is less than the density threshold, it is determined that the point cloud data block satisfies a second preset condition, and the process directly jumps to the data transmission process, and if the point density of the point cloud data block is greater than or equal to the density threshold, the point cloud data block needs to be downsampled to achieve the purpose of reducing the point density of the point cloud data block, so that the point cloud data block satisfies the second preset condition.

And c, performing downsampling processing on the point cloud data block at least once until the point cloud data block meets a second preset condition. In practical application, each pair of point cloud data blocks is subjected to one-time downsampling processing, so that the point density of the point cloud data blocks can be reduced, and the point cloud data blocks can reach a second preset condition by performing at least one-time downsampling processing on the point cloud data blocks.

In one embodiment, for one downsampling process, a target point is randomly determined from the point cloud data block, a search area is determined by taking the target point as a center of a circle and the downsampling radius as a radius of the circle, and a specified number of data points are randomly determined from the data points contained in the search area. Assuming that the downsampling radius is 5 and the specified number is 1, randomly determining one or more target points from the data points contained in the point cloud data block in one downsampling process, determining a circular area with the target point as a circle center and the radius of 5 as a search area for each target point, and randomly selecting 1 data point from the search area if the search area comprises n data points, thereby achieving the purpose of reducing the point density of the point cloud data block. Optionally, the point cloud data block after the first downsampling process may be judged again, and if the point cloud data block does not satisfy the second preset condition, the downsampling process may be performed again until the point density of the point cloud data block is smaller than the density threshold.

And d, determining each point cloud data block meeting the second preset condition as target point cloud data. In practical application, each point cloud data block can be subjected to down-sampling processing sequentially, and each point cloud data block can be subjected to down-sampling processing simultaneously in a parallel processing mode, so that the point cloud segmentation efficiency is improved.

The embodiment of the present invention further provides another point cloud segmentation method, which is applied to a server, wherein the server is in communication connection with a client, and the server is configured with a point cloud segmentation network, and referring to a flow schematic diagram of another point cloud segmentation method shown in fig. 2, the method mainly includes the following steps S202 to S206:

step S202, target point cloud data sent by a client side is received. In one embodiment, a point cloud segmentation request sent by a client may be accepted, where the point cloud segmentation request carries target point cloud data, and the point cloud segmentation request is an HTTP request.

And step S204, carrying out segmentation processing on the target point cloud data through the point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data. The point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data.

Step S206, the segmentation result is sent to the client.

The point cloud segmentation device provided by the embodiment of the invention can be used for segmenting target point cloud data obtained by a client based on the point cloud data to be segmented, so that the automation level of the segmented point cloud data can be obviously improved, and in addition, the point cloud segmentation network comprising a feature aggregation sub-network and a feature extraction sub-network is used for respectively extracting the structural features and the detail features of the target point cloud data, so that the segmentation result is obtained on the basis of the structural features and the detail features, and the accuracy of point cloud segmentation can be obviously improved.

In the point cloud segmentation method provided by the related art, the voxel-based segmentation model has the following disadvantages: the cost and memory footprint grows cubically with increasing input resolution, making it impossible to extend resolution. The following disadvantages exist for point-based segmentation models: at most 80% of the time is wasted in constructing sparse data with poor memory locality, rather than actual feature extraction. Based on this, the embodiment of the present invention provides an implementation manner for performing segmentation processing on target Point cloud data through a Point cloud segmentation network to obtain a segmentation result corresponding to the target Point cloud data, where the Point cloud segmentation network may adopt a PVCNN (Point-volume CNN) frame, so that the Point cloud segmentation network may perform efficient and fast 3D (3-dimension, three-dimensional) deep learning. The point cloud segmentation network has two branches, one of which is a feature aggregation sub-network, and is used for performing voxel-based feature aggregation, performing coarse-grained neighborhood feature extraction, and extracting an overall structure (i.e., structural features); one is a feature extraction sub-network, which directly performs feature conversion on target point cloud data by using an MLP (multi-layer perceptron), and is fine-grained point-by-point feature extraction for obtaining details; and finally, fusing the structural features and the detail features to obtain a segmentation result. See specifically step 1 through step 3 below:

step 1, converting the target point cloud data into voxel data through a feature aggregation sub-network, and performing feature aggregation processing on the voxel data to obtain the structural features of the target point cloud data. In practical applications, the core of convolution is to gather proximity information to extract local features, and based on the regularity of the voxels, the embodiment of the present invention chooses to perform feature aggregation in the voxels, see the schematic structural diagram of a point cloud segmentation network shown in fig. 3. On the basis of fig. 3, step 1 above may be performed with reference to steps 1.1 to 1.4 as follows:

step 1.1, normalization processing is carried out on the target point cloud data to obtain normalized data. Considering that the dimensions of different point clouds may be very different, the coordinates { pk } need to be normalized before converting the target point cloud data into voxel data. In an alternative embodiment, (1) a local coordinate system with the point center as the origin is configured, and all the points in the target point cloud data are converted based on the local coordinate system to obtain converted coordinates. (2) All points in the target point cloud data are normalized to a unit sphere by dividing all transformed coordinates by max | | | pk | | 2. (3) Zoom and convert the point to [0,1 ]. It should be noted that the point feature fk remains unchanged during the normalization process, and the normalized data is denoted ˆ pk.

And 1.2, converting the normalized data into voxel data according to a preset conversion formula. In one embodiment, the normalized data may be converted to voxel data as follows:

where r represents the voxel resolution, II.]Is a coordinateWhether it belongs to the binary progression of the voxel grid (u, v, w)A control indicator for the control of the position of the movable part,representing the c-th channel characteristic corresponding to the coordinates ˆ pk,is the normalization factor (i.e. the number of points falling on that voxel grid),、、when the equal sign is establishedBelonging to the voxel grid (u, v, w).

And step 1.3, performing characteristic aggregation processing on the voxel data to obtain a voxel characteristic corresponding to the voxel data. In practical applications, after converting the points to a voxel grid (i.e., the voxel data described above), 3D voxel convolution will be applied to aggregate the features. In addition, after processing the voxel data by using 3D voxel convolution, a batch normalization layer and a nonlinear activation function layer are added after the 3D voxel convolution, and the nonlinear activation function layer outputs the voxel characteristics corresponding to the voxel data.

And step 1.4, performing anti-voxel processing on the voxel characteristics to obtain the structural characteristics of the target point cloud data. Since the structural features and the detailed features need to be subjected to feature fusion, the voxel features need to be converted back to the point cloud domain. A straightforward implementation of voxel to point mapping is nearest neighbor interpolation, i.e. the assignment of voxel features to all points within a voxel. However, since this approach will cause points in the same voxel grid to always share the same features, the embodiment of the present invention performs inverse voxel processing on voxel features by using cubic linear interpolation to ensure that the features mapped to each point are different. Since the voxelization and de-voxelization processes involved in embodiments of the present invention are guided, the entire voxel-based feature aggregation sub-network can be optimized in an end-to-end manner.

And 2, performing feature extraction processing on each point in the target point cloud data through a feature extraction sub-network to obtain the detailed features of the target point cloud data. In one embodiment, with continued reference to FIG. 3, FIG. 3 also illustrates that the feature extraction sub-network includes multiple layers of perceptrons. Based on the feature extraction sub-network shown in fig. 3, when the detailed features of the target point cloud data are extracted, each point in the target point cloud data can be subjected to feature extraction processing through the multilayer perceptron, so that the detailed features of the target point cloud data are obtained. In practical applications, the voxel-based feature aggregation sub-network fuses neighborhood information at a coarse granularity, but in order to model fine-grained single-point features, determining the final segmentation result based on only low-resolution voxels may render the segmentation result less accurate. To this end, embodiments of the present invention operate directly on each point, extracting features of a single point with the MLP, each point of the MLP output having a distinct distinctive feature, thereby supplementing the coarse voxel-based information with this high-resolution single-point information.

And 3, performing feature fusion processing on the structural features and the detail features to obtain a segmentation result corresponding to the target point cloud data. In one embodiment, the structural features and the detail features may be subjected to feature fusion processing by using a bitwise addition method, so as to obtain a segmentation result corresponding to the target point cloud data.

According to the point cloud segmentation method provided by the embodiment of the invention, the target point cloud data is segmented by the point cloud segmentation network based on the PVCNN framework. The point cloud segmentation network takes point representation as 3D input data to reduce memory consumption, and meanwhile convolution is carried out in voxels to reduce irregularity and improve locality and sparse data access. The point cloud segmentation method provided by the embodiment of the invention is efficient in memory consumption and calculation speed. The embodiment of the invention evaluates the PVCNN on the semantic segmentation and component segmentation data sets, the PVCNN performance is higher than the reference based on the voxel, and the memory of a GPU (graphics processing unit) is reduced to 10%; in addition, PVCNN achieves an average 7-fold acceleration of operation compared to point-based methods.

To facilitate understanding of the point cloud segmentation method provided in the foregoing embodiment, an application example of the point cloud segmentation method is provided in the embodiment of the present invention, referring to a flowchart of another point cloud segmentation method shown in fig. 4, where the method mainly includes the following steps S402 to S424:

step S402, the client reads the point cloud data to be segmented. For example, the point cloud data to be segmented is read from a specified storage path by using a "read _ file ()" instruction.

In step S404, the client identifies a file suffix of the point cloud data to be segmented. If the file suffix is txt suffix, executing step S406; if the file suffix is a last suffix, step S408 is performed.

In step S406, the client reads the point cloud data to be segmented in txt format.

Step S408, the client reads the point cloud data to be segmented in the copy format.

And step S410, the client performs block cutting on the point cloud data to be segmented. For example, the point cloud data to be segmented is segmented by using a "crop _ and _ save ()" instruction, wherein the default step size (i.e., the preset step size) is 100.

In step S412, the client performs downsampling on the point cloud data to be segmented to obtain target point cloud data. For example, the point cloud data to be segmented is downsampled by using a "downsampling ()" instruction, where a parameter voxel _ size of the downsampling is 0.2.

In step S414, the client sends an http request to the server. And the http request carries target point cloud data.

In step S416, the server receives the http request back.

Step S418, the server analyzes the data to obtain target point cloud data.

Step S420, the server inputs the target point cloud data into the point cloud segmentation network to obtain a segmentation result. The point cloud segmentation network adopts a PVCNN framework.

In step S422, the server feeds back the segmentation result to the client.

In step S424, the client saves the segmentation result. For example, the split result is saved with a "save _ data-fun ()" instruction.

In summary, the point cloud segmentation method provided by the embodiment of the present invention at least has the following characteristics:

(1) easy to use, requires less expertise: when the system is used, only the storage path needs to be appointed, and the client can automatically segment the point cloud data.

(2) The segmentation accuracy is high: if the experience of the professionals is finely divided, the traditional method needs to consume higher time cost, but the point cloud segmentation program iou (interaction over Union) provided by the embodiment of the invention can reach 93%, so that the workload of the professionals is greatly reduced.

(3) Software iterative updates are easier: the point cloud segmentation program uses the design of front-end and back-end classification, the point cloud segmentation network is configured at the server, the point cloud segmentation network is prevented from being changed when the customization function is added subsequently, and the use of the client is not influenced when the point cloud segmentation network is updated to improve the accuracy.

As to the point cloud segmentation method provided in the foregoing embodiment, an embodiment of the present invention provides a point cloud segmentation apparatus, which is applied to a client, where the client is in communication connection with a server, and the server is configured with a point cloud segmentation network, and referring to a schematic structural diagram of the point cloud segmentation apparatus shown in fig. 5, the apparatus mainly includes the following components:

the data reading module 502 is configured to, if a point cloud segmentation request is received, call a preset interface to read point cloud data to be segmented;

a judging module 504, configured to judge whether the point cloud data to be segmented satisfies a first preset condition based on the number of points of the point cloud data to be segmented;

the block cutting module 506 is configured to, when the judgment result of the judgment module is yes, perform block cutting on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets a first preset condition;

a down-sampling module 508, configured to down-sample each point cloud data block to obtain target point cloud data;

a data sending module 510, configured to send the target point cloud data to a server, so that the server performs segmentation processing on the target point cloud data by using a point cloud segmentation network to obtain a segmentation result;

and a result receiving module 512, configured to receive the segmentation result fed back by the server.

The point cloud segmentation device provided by the embodiment of the invention can automatically execute the operations of reading, blocking processing, down-sampling processing, data sending, result receiving and the like of point cloud data to be segmented after receiving a point cloud segmentation request, so that the automation level of the segmented point cloud data is obviously improved, a point cloud segmentation network configured in a server is used for segmenting target point cloud data, a final segmentation result is obtained based on structural features and detail features, the accuracy of point cloud segmentation can be obviously improved, a user does not need to adjust the segmentation result again, and the dependence of a point cloud segmentation task on the experience of the user can be effectively reduced.

In one embodiment, the down-sampling module 508 is further configured to: acquiring preset slicing parameters, and carrying out slicing processing on point cloud data to be segmented based on the slicing parameters to obtain a plurality of point cloud data blocks; judging whether each point cloud data block meets a first preset condition; and if the point cloud data blocks do not meet the first preset condition, carrying out block cutting processing on the point cloud data blocks based on the block cutting parameters until each point cloud data block meets the first preset condition.

In one embodiment, the dicing module 506 is further configured to: acquiring a preset down-sampling parameter; wherein the down-sampling parameter comprises a down-sampling radius; for each point cloud data block, judging whether the point cloud data block meets a second preset condition or not based on the point density of the point cloud data block; if not, performing at least one time of downsampling processing on the point cloud data block until the point cloud data block meets a second preset condition; and determining each point cloud data block meeting the second preset condition as target point cloud data.

In one embodiment, the dicing module 506 is further configured to: for one-time downsampling processing, a target point is randomly determined from the point cloud data block, a search area is determined by taking the target point as a circle center and the downsampling radius as a circle radius, and a specified number of data points are randomly determined from the data points contained in the search area.

For the point cloud segmentation method provided in the foregoing embodiment, an embodiment of the present invention provides another point cloud segmentation apparatus, where the apparatus is applied to a server, the server is in communication connection with a client, and the server is configured with a point cloud segmentation network, and referring to a schematic structural diagram of another point cloud segmentation apparatus shown in fig. 6, the apparatus mainly includes the following components:

a data receiving module 602, configured to receive target point cloud data sent by a client; the target point cloud data is obtained by a client based on the point cloud data to be segmented, and the client is used for calling a preset interface to read the point cloud data to be segmented when receiving a point cloud segmentation request; judging whether the point cloud data to be segmented meet a first preset condition or not based on the number of points of the point cloud data to be segmented; if not, performing block cutting on the point cloud data to be segmented to obtain a plurality of point cloud data blocks until each point cloud data block meets a first preset condition; respectively performing down-sampling processing on each point cloud data block to obtain target point cloud data;

the segmentation module 604 is configured to perform segmentation processing on the target point cloud data through a point cloud segmentation network to obtain a segmentation result corresponding to the target point cloud data; the point cloud segmentation network comprises a feature aggregation sub-network and a feature extraction sub-network, wherein the feature aggregation sub-network is used for extracting the structural features of the target point cloud data, and the feature extraction sub-network is used for extracting the detailed features of the target point cloud data;

a result sending module 606, configured to send the segmentation result to the client.

The point cloud segmentation device provided by the embodiment of the invention can be used for segmenting target point cloud data obtained by a client based on the point cloud data to be segmented, so that the automation level of the segmented point cloud data can be obviously improved, and in addition, the point cloud segmentation network comprising a feature aggregation sub-network and a feature extraction sub-network is used for respectively extracting the structural features and the detail features of the target point cloud data, so that the segmentation result is obtained on the basis of the structural features and the detail features, and the accuracy of point cloud segmentation can be obviously improved.

In one embodiment, the segmentation module 604 is further configured to: converting the target point cloud data into voxel data through a feature aggregation sub-network, and performing feature aggregation processing on the voxel data to obtain structural features of the target point cloud data; performing feature extraction processing on each point in the target point cloud data through a feature extraction sub-network to obtain detailed features of the target point cloud data; and carrying out feature fusion processing on the structural features and the detail features to obtain a segmentation result corresponding to the target point cloud data.

In one embodiment, the segmentation module 604 is further configured to: carrying out normalization processing on the target point cloud data to obtain normalized data; converting the normalized data into voxel data according to a preset conversion formula; performing characteristic aggregation processing on the voxel data to obtain a voxel characteristic corresponding to the voxel data; and performing inverse voxel processing on the voxel characteristics to obtain the structural characteristics of the target point cloud data.

In one embodiment, the feature extraction subnetwork comprises a multi-layer perceptron; the segmentation module 604 is further configured to: and performing feature extraction processing on each point in the target point cloud data through a multilayer perceptron to obtain the detail features of the target point cloud data.

The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments.

The embodiment of the invention provides terminal equipment, which particularly comprises a processor and a storage device; the storage means has stored thereon a computer program which, when executed by the processor, performs the method of any of the above embodiments.

Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present invention, where the terminal device 100 includes: a processor 70, a memory 71, a bus 72 and a communication interface 73, wherein the processor 70, the communication interface 73 and the memory 71 are connected through the bus 72; the processor 70 is arranged to execute executable modules, such as computer programs, stored in the memory 71.

The Memory 71 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.

The bus 72 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.

The memory 71 is configured to store a program, and the processor 70 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 70, or implemented by the processor 70.

The processor 70 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 70. The Processor 70 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 71, and the processor 70 reads the information in the memory 71 and completes the steps of the method in combination with the hardware thereof.

The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the foregoing method embodiment, which is not described herein again.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.