阅读说明：本技术 一种光栅投影轮廓有效点自动提取方法及系统 (Automatic extraction method and system for effective points of grating projection profile ) 是由 马峻 陈宏� 陈寿宏 郭玲 徐翠锋 于 2019-09-26 设计创作，主要内容包括：本发明公开了一种光栅投影轮廓有效点自动提取方法及系统,方法包括以下步骤：在PC机上编码生成正弦条纹图像；投影仪将所述正弦条纹图像投影到被测物表面；CMOS相机采集所述被测物的多幅调制图像；通过软件编程解包裹相位,重建所述被测物的三维轮廓；其中,在获取所述调制图像以后,把像素点聚类设置为背景点、边界点与目标点三类,并分别确定边界点初始质心、目标点初始质心和背景点初始质心；进行聚类处理,定义一个阈值tm,第t次聚类时,质心前后两次迭代的质心值之间的差值小于阈值tm时,即可认为此时的质心趋近于收敛；如果目标点质心趋近于收敛,则把大于c3t的点移除数据列表；如果背景点质心趋近于收敛,则把小于c1t的点移除数据列表。(The invention discloses a method and a system for automatically extracting effective points of a grating projection contour, wherein the method comprises the following steps: coding on a PC to generate a sine stripe image; the projector projects the sine stripe image to the surface of the measured object; the CMOS camera collects a plurality of modulation images of the measured object; unwrapping the phase by software programming, and reconstructing a three-dimensional profile of the measured object; after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points; clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence; removing points greater than c3t from the data list if the target point centroid approaches convergence; if the background point centroid approaches convergence, then points less than c1t are removed from the data list.)

1. A method for automatically extracting effective points of a grating projection profile is characterized by comprising the following steps:

coding on a PC to generate a sine stripe image;

the projector projects the sine stripe image to the surface of the measured object;

the CMOS camera collects a plurality of modulation images of the measured object;

unwrapping the phase by software programming, and reconstructing a three-dimensional profile of the measured object; wherein the content of the first and second substances,

after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points;

clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence;

removing points greater than c3t from the data list if the target point centroid approaches convergence;

if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

2. The method for automatically extracting the effective points of the projection profile of the grating as claimed in claim 1, wherein: the modulation image is obtained according to the following formula:

where N is the nth phase shift, N represents the total phase shift, and In is the fringe intensity plot of the nth phase shift.

3. The method for automatically extracting the valid points of the projection profile of the grating as claimed in claim 2, wherein the three initial centroids are determined according to the following formula:

wherein the boundary point initial centroid c2 is the average modulation value of the entire modulation map, the target point initial centroid c3 is the average of all points greater than c2, and the background point initial centroid c1 is the average of all points less than centroid c 2.

4. The method for automatically extracting the effective points of the projection profile of the grating as claimed in claim 3, wherein: after the clustering is completed, the points of the boundary points can be divided into target points if they satisfy one of the following two conditions:

first, the point is directly connected to the target point;

second, the gradient at this point is less than 2 times the maximum gradient at the target point.

5. The method for automatically extracting the effective points of the projection profile of the grating as claimed in claim 4, wherein: and segmenting background points in the modulated image after clustering is finished by using an Otsu method, and taking a part with a large threshold value in a segmentation result as a target point.

6. The method for automatically extracting the effective points of the projection profile of the grating as claimed in claim 5, wherein: the CMOS camera collects four modulation images of the measured object.

7. An automatic extraction system for effective points of a grating projection profile, which is characterized by comprising:

the PC is used for encoding to generate a sine stripe image;

the projector is used for projecting the sine stripe image to the surface of the measured object; and

the CMOS camera is used for acquiring a plurality of modulation images of the measured object; wherein the content of the first and second substances,

the PC is also used for unwrapping the phase through software programming and reconstructing the three-dimensional profile of the measured object; after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points; clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence; removing points greater than c3t from the data list if the target point centroid approaches convergence; if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

8. The system according to claim 7, wherein the system comprises: the CMOS camera acquires four modulated images of the measured object.

Technical Field

The invention relates to the technical field, in particular to a method and a system for automatically extracting effective points of a grating projection profile.

Background

In the grating projection technology, the phase shift method is the most common method for measuring three dimensions, and the method uses a projector to project grating stripes onto a measured object, then uses a camera to shoot a grating pattern of the measured object, and a computer analyzes the grating pattern and reconstructs the surface geometry of the measured object according to the analysis result and the parameters of the system. For some measured objects, the grating stripes can not completely cover the surface of the measured object, so that a shadow area exists in a picture captured by a camera. The shadow area does not carry any information related to the surface of the measured object, and the three-dimensional reconstruction result of the area is wrong. The shadow area needs to be removed, except for the shadow area, the background points in the grating projection system also need to be removed, only the effective area of the object to be measured is reserved, and the result of three-dimensional reconstruction is optimal.

The prior similar implementation schemes are as follows:

skydan uses multiple projectors to illuminate the object under test from different viewing angles to obtain shadowless three-dimensional reconstruction data

Zhang et al removes random noise with a gaussian filter and uses the characteristics of phase monotonicity to identify phase null points.

Xiao proposes a method for dividing the target and background, which effectively removes the invalid point, but needs to separately obtain the modulation image of the background plate first, then put the target into the grating projection system to obtain the modulation image of the target and the background plate

And 4, Wang proposes that pixel points are divided into three types of points, namely target points, boundary points and background points in a modulation graph, clustering is carried out by utilizing a K-means method, and after clustering is finished, each boundary point is judged to belong to a target point or a background point, so that the target points can be automatically and effectively extracted.

However, the above methods all have some problems:

the Skydan approach requires multiple cameras, additional cost in hardware, making it less popular

The Zhang method works well only on objects with flat surfaces, and when the object has a surface that changes rapidly, the gaussian filter treats the surface as noise

The method of Xiao requires cumbersome operations

The Wang method has the problems that clustering iteration time is too long, target points exist in background points and the like.

Disclosure of Invention

In view of the above, the present invention provides a method and a system for automatically extracting effective points of a grating projection profile, which reduce iterative convergence time of a conventional K-means method, and obtain more target points by re-segmenting background points by combining an Otsu method, so as to achieve fast, automatic, efficient, simple, convenient and economical extraction of effective points of a measured object.

The invention solves the technical problems by the following technical means:

a method for automatically extracting effective points of a grating projection profile comprises the following steps:

coding on a PC to generate a sine stripe image;

the projector projects the sine stripe image to the surface of the measured object;

the CMOS camera collects a plurality of modulation images of the measured object;

unwrapping the phase by software programming, and reconstructing a three-dimensional profile of the measured object; wherein the content of the first and second substances,

after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points;

clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence;

removing points greater than c3t from the data list if the target point centroid approaches convergence;

if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

Further, the modulation image is obtained according to the following formula:

where N is the nth phase shift, N represents the total phase shift, and In is the fringe intensity plot of the nth phase shift.

Further, the three initial centroids are determined according to the following formula:

wherein the boundary point initial centroid c2 is the average modulation value of the entire modulation map, the target point initial centroid c3 is the average of all points greater than c2, and the background point initial centroid c1 is the average of all points less than centroid c 2.

Further, after the clustering is completed, the point of the boundary point can be divided into target points if it satisfies one of the following two conditions:

first, the point is directly connected to the target point;

second, the gradient at this point is less than 2 times the maximum gradient at the target point.

Further, segmentation is carried out on background points in the modulated image after clustering is completed by using an Otsu method, and a part with a large threshold value in a segmentation result is taken as a target point.

Further, the CMOS camera collects four modulated images of the measured object.

On the other hand, the invention also provides an automatic extraction system for effective points of a grating projection profile, which comprises the following components:

the PC is used for encoding to generate a sine stripe image;

the projector is used for projecting the sine stripe image to the surface of the measured object; and

the CMOS camera is used for acquiring a plurality of modulation images of the measured object; wherein the content of the first and second substances,

the PC is also used for unwrapping the phase through software programming and reconstructing the three-dimensional profile of the measured object; after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points; clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence; removing points greater than c3t from the data list if the target point centroid approaches convergence; if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

Furthermore, the modulated images of the measured object collected by the CMOS camera are four.

The invention has the beneficial effects that: the invention improves the conventional K-means method, reduces clustering iteration convergence time, and the result obtained by the improved method is consistent with the result obtained by the conventional method, but only needs less running time, and in 1024 x 1280 pictures, the running time of the conventional K-means is 32.214 seconds; the improved K-menas process of the present invention is 18.741 seconds. Further, the background points in the conventional K-means have wrongly-divided target points, and are not processed, and the Otsu method is used for carrying out secondary processing on the background points to obtain more effective points.

Drawings

Fig. 1 is a schematic structural diagram of an automatic extraction system for effective points of a grating projection profile according to an embodiment of the present invention;

fig. 2 is a flowchart of a method for automatically extracting valid points of a grating projection profile according to an embodiment of the present invention;

fig. 3 is a diagram of a moving process of an initial centroid of a boundary point, an initial centroid of a target point, and an initial centroid of a background point in the method for automatically extracting effective points of a grating projection profile provided in the embodiment of the present invention;

fig. 4 is a modulation image provided by an embodiment of the present invention for verifying the effect of the present invention;

FIG. 5 is a graph of the results of the manual segmentation applied to FIG. 4;

FIG. 6 is a graph showing the results of the conventional K-means segmentation applied to FIG. 4;

FIG. 7 is a graph comparing the differences between FIG. 6 and FIG. 5;

FIG. 8 is a graph of the segmentation results of FIG. 4 using a method according to an embodiment of the present invention;

fig. 9 is a graph comparing the difference between fig. 8 and fig. 5.

Detailed Description

The invention will be described in detail below with reference to the following figures and specific examples:

as shown in fig. 2, the method for automatically extracting effective points of a grating projection profile of the present invention includes the following steps:

coding on a PC to generate a sine stripe image;

the projector projects the sine stripe image to the surface of the measured object;

the CMOS camera collects a plurality of modulation images of the measured object;

unwrapping the phase by software programming, and reconstructing a three-dimensional profile of the measured object; wherein the content of the first and second substances,

after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points;

clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence;

removing points greater than c3t from the data list if the target point centroid approaches convergence;

if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

Specifically, the CMOS camera acquires four modulated images of the measured object, the invention removes the invalid point and uses the modulated image of the measured object, and the modulated image acquisition method is as shown in formula (1):

where N is the nth phase shift, N represents the total phase shift, and In is the fringe intensity plot of the nth phase shift.

After the modulation map is acquired, clustering is performed using the improved K-means algorithm of the present invention. The invention sets the pixel point cluster as three types (K is 3) of background point, boundary point and target point, the initial centroid c2 of the boundary point is the average modulation value of the whole modulation chart, the initial centroid c3 of the target point is the average value of all points which are larger than c2, and the initial centroid c1 of the background point is the average value of all points which are smaller than the centroid c 2. The determination of the three initial centroids is shown in equation (2):

the clustering is started after the centroid is set, and the innovation points of the invention are as follows: in the clustering process, a threshold value tm is defined, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, the centroid at the moment can be considered to approach convergence. If the centroid of the target point S3 approaches convergence, points greater than c3t are removed from the data list. If the centroid of the background point S2 is close to convergence, the points smaller than c1t are removed from the data list, so that the number of clustering points can be reduced, and the clustering iteration time is shortened. The process of moving the respective centroids is shown in fig. 3.

The clustering result of the method is consistent with the traditional K-means result, but the required time is shorter. In the plot of plot 41024 x 1280, the clustering results are shown in table 1:

table 1 difference between the algorithm of the present invention and the conventional algorithm

After clustering is completed, the three points are divided into a target point, a background point and a boundary point. Boundary point S₂The point of (2) satisfying one of the following two conditions can be classified as a target point, and first, the point is directly connected to the target point. Second, the gradient at this point is less than 2 times the maximum gradient at the target point. In order to obtain more target pointsEight neighborhood points of the newly added target point are also divided into target points.

In the background points after clustering, there are target points, fig. 5 is a segmentation result of manually selecting a threshold, fig. 6 is a clustering result, and fig. 7 is a difference of fig. 5 from fig. 6.

The right ear and cheek portion dots are background dots in fig. 6, and are target dots in fig. 5. These modulation values were observed to be 12.7-14.5, while the modulation values for the true background points were 0.5-3.5. Therefore, the background point can be divided by using the Otsu method, and the division node can be taken

The part of the result where the threshold is large is taken as the target point. FIG. 8 shows the results obtained by the method of the present invention, and FIG. 9 shows the difference between the method of the present invention and the manual segmentation. Obviously, the difference between the method and the manual segmentation is smaller, the result is better than that obtained by the conventional K-means, the number of the effective points of the manual segmentation result is 390703 and accounts for 29.80 percent of the total points, the number of the effective points of the traditional K-means clustering result is 363461 and accounts for 27.75 percent of the total points, and the number of the effective points of the method is 391450 and accounts for 29.86 percent of the total points.

On the other hand, as shown in the figure, an embodiment of the present invention further provides an automatic extraction system for effective points of a grating projection profile, where the system includes:

the PC is used for encoding to generate a sine stripe image;

the projector is used for projecting the sine stripe image to the surface of the measured object; and

the CMOS camera is used for acquiring a plurality of modulation images of the measured object; wherein the content of the first and second substances,

the PC is also used for unwrapping the phase through software programming and reconstructing the three-dimensional profile of the measured object; after the modulation image is obtained, clustering pixel points into three types of background points, boundary points and target points, and respectively determining an initial centroid of the boundary points, an initial centroid of the target points and an initial centroid of the background points; clustering, defining a threshold value tm, and when the difference value between the centroid values of two iterations before and after the centroid is smaller than the threshold value tm in the t-th clustering, determining that the centroid at the moment approaches convergence; removing points greater than c3t from the data list if the target point centroid approaches convergence; if the background point centroid approaches convergence, then points less than c1t are removed from the data list.

Specifically, the CMOS camera acquires four modulated images of the object to be measured.

Although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims. The techniques, shapes, and configurations not described in detail in the present invention are all known techniques.