阅读说明：本技术 一种级次线配对的无相位展开三维人脸重建方法及系统 (Phase-unfolding-free three-dimensional face reconstruction method and system for level line pairing ) 是由 郭燕琼 游志胜 吕坤 朱江平 刘波 于 2021-09-07 设计创作，主要内容包括：本发明涉及光学三维成像领域,特别涉及一种级次线配对的无相位展开三维人脸重建方法及系统。本发明通过调整特征点,并以特征点为基础,采用在人脸上画线的方式来辅助截断相位级次线进行配对；可迅速、准确的实现各人脸图像上的截断相位级次线匹配对；同时在截断相位级次线的配对时无需借助其它辅助信号(如散斑、三角波等),降低了对投影光场的要求；本发明使用视差匹配矩阵指导截断相位图的匹配过程,直接得到稠密视差图,并根据稠密视差图,重建三维人脸；大大提高了三维重建的精度、准确度。(The invention relates to the field of optical three-dimensional imaging, in particular to a phase-unwrapped three-dimensional face reconstruction method and system based on rank line pairing. The invention adjusts the characteristic points and adopts a mode of drawing lines on the face to assist in truncating the phase level lines for matching on the basis of the characteristic points; the matching pair of the truncated phase level lines on each face image can be rapidly and accurately realized; meanwhile, other auxiliary signals (such as speckles, triangular waves and the like) are not needed when pairing of the phase-cut secondary lines is cut off, and the requirement on a projection light field is lowered; the method uses the parallax matching matrix to guide the matching process of the truncated phase diagram, directly obtains a dense parallax diagram, and reconstructs a three-dimensional face according to the dense parallax diagram; the precision and accuracy of three-dimensional reconstruction are greatly improved.)

1. A phase-unwrapped three-dimensional face reconstruction method of a secondary line pairing is characterized by comprising the following steps:

s1, respectively collecting face images of N frames of detected faces in a structured light field at different M shooting angles; performing epipolar line correction on the acquired light field image of the structure; wherein N is an integer of 3 or more, and M is an integer of 2 or more;

s2, respectively resolving a truncation phase in the structure light field image after polar line correction to obtain a truncation phase diagram; obtaining a truncated phase level line by truncating the phase; analyzing texture images in the structured light field images after polar line correction, and extracting characteristic points; selecting and aligning the characteristic points to obtain a reference point group;

s3, matching the truncated phase level secondary line through the reference point group to obtain a truncated phase level secondary line group; matching all pixel points in the same truncated phase level sub-line group by using a search algorithm and marking level serial numbers;

s4, obtaining a parallax matching matrix through matching of the truncated phase level secondary line groups;

and S5, matching the truncated phase images by using the parallax matching matrix to obtain a dense parallax image, and reconstructing a three-dimensional face according to the dense parallax image.

2. The method for reconstructing three-dimensional human face without phase unwrapping of secondary line pair as claimed in claim 1, wherein the step S2 includes:

s21, correspondingly selecting T reference points from the feature points extracted from the different texture images, wherein T is more than or equal to 2;

and S22, correspondingly moving the selected reference points to the secondary line and carrying out pairing calculation to obtain a reference point group.

3. The method of claim 2, wherein the fiducial points comprise: human face characteristic points and/or human face characteristic point derivative points;

the human face feature points include: eye, eyebrow, nose tip, corner of mouth, or cheek contour feature points; the face characteristic point derivative point is the middle point of two face characteristic points or the center of a plurality of face characteristic points.

4. The method for reconstructing a three-dimensional face without phase unwrapping by using a secondary line pair as claimed in claim 1, wherein the step S3 specifically includes:

s31, drawing lines on the face image according to a preset path by taking the reference point group as a reference, marking and sequencing intersection points of the drawn lines and the secondary lines, and arranging the serial numbers of the secondary lines according to the intersection points; placing the same serial number of the secondary lines into a truncated phase-level secondary line group;

and S32, marking the corresponding level serial number of each pixel point on each level line in the same truncated phase level line group through a search algorithm.

5. The method for reconstructing a three-dimensional face without phase unwrapping by using a secondary line pair as claimed in claim 1, wherein the step S4 specifically includes:

searching pixels with the same sequence number as the pixel on the same line on other secondary lines of a truncated phase secondary line group where the pixel is located by taking the pixel on the secondary line in any image as a reference, extracting the vertical coordinate of the pixels, and adding the pixels into a parallax matching matrix; if the same row does not search for the same order number, 0 is placed in the disparity matching matrix.

6. The method for reconstructing a three-dimensional human face without phase unwrapping of any one of claims 1-5, wherein the truncated phase level is a curve formed by jumping pixel points in truncated phase information; and the hopping pixel points are pixel points which hop the phase value in the truncated phase diagram from pi to pi or from-pi to pi.

7. The method of claim 6, wherein the structured light field comprises a sine-striped structured light field.

8. A system for phase-unwrapped three-dimensional face reconstruction from a pair of secondary lines, the system comprising: the system comprises a light field projection device, M cameras and a control module;

the control module is configured to send a control signal to a light field projection device, so that the light field projection device projects a light field sequence with a sine stripe structure and an adjustable number of images onto the surface of a human face, and the three-dimensional human face reconstruction method according to any one of claims 1 to 7 is executed based on a human face image transmitted by a camera to complete three-dimensional human face reconstruction;

the light field projection device is used for projecting the sinusoidal stripe structure light field sequence with adjustable image quantity to the surface of a human face, and when receiving a control signal sent by the control module, the light field projection device performs N frames of stripe structure light projection and sends N synchronous control signals to the M cameras;

the M cameras are used for shooting multi-angle images of the face surface under the illumination of the fringe structure light field under the control of a synchronous control signal and transmitting the shot face images to the control module;

wherein N is an integer of 3 or more, and M is an integer of 2 or more.

9. The system of claim 8, wherein the control module is one of a controller, a processor, a single chip or a PC with signal and data processing capabilities.

10. A readable storage medium having stored thereon a computer program for executing a method for phase-unwrapped three-dimensional face reconstruction by implementing a secondary line pairing as claimed in any one of claims 1 to 7.

Technical Field

The invention relates to the field of optical three-dimensional imaging, in particular to a phase-unwrapped three-dimensional face reconstruction method and system based on rank line pairing.

Background

The premise of the rapid application of the three-dimensional face recognition technology is the large-scale three-dimensional face data library establishment, and the optical three-dimensional measurement technology based on the triangulation principle has the remarkable advantages of full-field non-contact, high precision, high speed and the like, and is considered to be one of the important advocating technologies for acquiring high-speed high-precision three-dimensional face data. Three-dimensional surface shape data is obtained by projecting a structured light field to the surface of a face to be measured, a monocular or binocular camera is usually adopted to shoot a deformation image sequence modulated by the surface of the face to be measured, and phase information is extracted to reconstruct a three-dimensional model. The purpose of the structured light coding is to enrich or increase the characteristics of the surface of the detected weak texture face, so that the accuracy and reliability of a three-dimensional reconstruction result and the integrity of modeling are improved. At present, structured light coding mainly comprises speckle structured light and stripe structured light, wherein a binocular stereo matching system based on the stripe structured light coding is widely applied due to the obvious advantage of high precision. In fringe structured light systems, phase-shift profilometry (PSP) is known for its higher accuracy, greater resolution, lower complexity and insensitivity to ambient light.

PSP is applied on the premise that the measured object is relatively stationary. However, unlike measuring static subjects, the face is more or less in motion, e.g., breathing, blinking, twitching or having other dynamic expressions. In the dynamic three-dimensional face recognition process, besides the precision requirement, it is also desirable to obtain a real-time three-dimensional face model. Therefore, for a high-precision sine stripe structured light three-dimensional face reconstruction system, the following two challenges are also faced:

(1) the number of image acquisition should be reduced as much as possible, and the acquisition time should be saved. In the traditional method, the number of projection stripes is large, such as a multi-frequency method, a Gray code method and the like, in the phase resolving process, a plurality of frames of low-frequency signals are projected to resolve a high-frequency signal to obtain a high-precision absolute phase, so that the acquisition time is greatly increased, and the modeling precision is reduced due to artifacts caused by the motion of the human face.

(2) The reconstruction algorithm is optimized as much as possible, and the calculation time is saved. The most popular phase unwrapping algorithms currently exist in two categories: a spatial phase unwrapping algorithm and a temporal phase unwrapping algorithm. The spatial phase unwrapping algorithm firstly determines 2 pi discontinuous positions on a truncated phase diagram, and then deletes the discontinuity by adding or subtracting integer multiples of 2 pi, but the spatial phase unwrapping algorithm has the problem of poor robustness caused by unwrapping path error accumulation. Although the time phase expansion can overcome the difficulty of the space phase expansion well, and many methods such as a multi-frequency method and a gray code method have been developed over the years, the number of fringe projections is still the problem to be solved at present.

For example, the chinese patent publication No. CN109903377A proposes a three-dimensional face modeling method and system without phase expansion, in which the method uses face feature points to generate face geometric information constraint conditions, and uses the face geometric information constraint conditions to constrain the binocular stereo matching process; the truncation phase can be directly used for carrying out three-dimensional face reconstruction; however, the method simply carries out stereo matching through the constraint condition of the geometric information of the human face, and has the defects of low matching precision and poor three-dimensional reconstruction effect.

Disclosure of Invention

At least one of the objectives of the present invention is to provide a method and a system for reconstructing a three-dimensional face without phase unwrapping based on rank matching, so as to overcome the problem of low accuracy of the conventional three-dimensional face reconstruction without phase unwrapping.

The first aspect of the present application provides a method for reconstructing a three-dimensional face without phase unwrapping of a secondary line pair, where the method may be executed by a control device or may be executed by a chip configured in the control device, and the present application is not limited thereto. The method comprises the following steps:

s1, respectively collecting face images of N frames of detected faces in a structured light field at different M shooting angles; performing epipolar line correction on the acquired light field image of the structure; wherein N is an integer of 3 or more, and M is an integer of 2 or more;

s2, respectively resolving a truncation phase in the structure light field image after polar line correction to obtain a truncation phase diagram; obtaining a truncated phase level line by truncating the phase; analyzing texture images in the structured light field images after polar line correction, and extracting characteristic points; selecting and aligning the characteristic points to obtain a reference point group;

s3, matching the truncated phase level secondary line through the reference point group to obtain a truncated phase level secondary line group; matching all pixel points in the same truncated phase level sub-line group by using a search algorithm and marking level serial numbers;

s4, obtaining a parallax matching matrix through matching of the truncated phase level secondary line groups;

and S5, matching the truncated phase images by using the parallax matching matrix to obtain a dense parallax image, and reconstructing a three-dimensional face according to the dense parallax image.

Further, step S2 includes:

s21, correspondingly selecting T reference points from the feature points extracted from the different texture images, wherein T is more than or equal to 2;

and S22, correspondingly moving the selected reference points to the secondary line and carrying out pairing calculation to obtain a reference point group.

Further, the reference points include: human face characteristic points and/or human face characteristic point derivative points;

the human face feature points include: eye, eyebrow, nose tip, corner of mouth, or cheek contour feature points; the face characteristic point derivative point is the middle point of two face characteristic points or the center of a plurality of face characteristic points.

Further, the step S3 specifically includes:

s31, drawing lines on the face image according to a preset path by taking the reference point group as a reference, marking and sequencing intersection points of the drawn lines and the secondary lines, and arranging the serial numbers of the secondary lines according to the intersection points; placing the same serial number of the secondary lines into a truncated phase-level secondary line group;

and S32, marking the corresponding level serial number of each pixel point on each level line in the same truncated phase level line group through a search algorithm.

Further, the step S4 specifically includes:

searching pixels with the same sequence number as the pixel on the same line on other secondary lines of a truncated phase secondary line group where the pixel is located by taking the pixel on the secondary line in any image as a reference, extracting the vertical coordinate of the pixels, and adding the pixels into a parallax matching matrix; if the same row does not search for the same order number, 0 is placed in the disparity matching matrix.

It should be noted that the truncation phase level line is a curve formed by the jumping pixels in the truncation phase information; and the hopping pixel points are pixel points which hop the phase value in the truncated phase diagram from pi to pi or from-pi to pi.

In one possible embodiment of the present application, the structured light field comprises a sinusoidal fringe structured light field.

The second aspect of the present invention provides a phase-unwrapped three-dimensional face reconstruction system with a secondary line pair, comprising: the system comprises a light field projection device, M cameras and a control module;

the control module is configured to send a control signal to a light field projection device, so that the light field projection device projects a light field sequence with a sinusoidal stripe structure, the number of images of which is adjustable, onto a face surface, and on the basis of a face image transmitted by a camera, any one of possible implementation manners in a phase-unfolding-free three-dimensional face reconstruction method with a matched rank line provided by the first aspect of the present application is executed;

the light field projection device is used for projecting the sinusoidal stripe structure light field sequence with adjustable image quantity to the surface of a human face, and when receiving a control signal sent by the control module, the light field projection device performs N frames of stripe structure light projection and sends N synchronous control signals to the M cameras;

the M cameras are used for shooting multi-angle images of the face surface under the illumination of the fringe structure light field under the control of a synchronous control signal and transmitting the shot face images to the control module;

wherein N is an integer of 3 or more, and M is an integer of 2 or more.

A second aspect of the present invention provides a readable storage medium, on which a computer program is stored, where the program is executed by a processor to implement any one of the possible embodiments of the phase unwrapped three-dimensional face reconstruction method for secondary line pairing provided in the first aspect of the present application.

In summary, compared with the prior art, the invention has the following beneficial effects:

1. according to the invention, three-dimensional reconstruction can be completed by projecting N frames of phase-shifted sine stripe structured light at least and without projecting information of an additional structured light field or embedding other auxiliary signals (such as speckles, triangular waves and the like), so that the image acquisition time is saved, and the dynamic measurement sensitivity is reduced; meanwhile, three-dimensional reconstruction can be realized without phase expansion, and the time required by three-dimensional reconstruction is reduced under the condition of not improving the calculation force; meanwhile, compared with the existing phase-free unfolding method, the method provided by the invention has the characteristics of high robustness, high precision and high speed;

2. the invention adjusts the characteristic points and adopts a mode of drawing lines on the face to assist in truncating the phase level lines for matching on the basis of the characteristic points; the matching pair of the truncated phase level lines on each face image can be rapidly and accurately realized; meanwhile, other auxiliary signals (such as speckles, triangular waves and the like) are not needed when pairing of the phase-cut secondary lines is cut off, and the requirement on a projection light field is lowered;

3. the method uses the parallax matching matrix to guide the matching process of the truncated phase diagram, directly obtains a dense parallax diagram, and reconstructs a three-dimensional face according to the dense parallax diagram; the precision and accuracy of three-dimensional reconstruction are greatly improved.

Description of the drawings:

fig. 1 is a schematic diagram of a three-dimensional face reconstruction system with no phase unwrapping for a secondary line pairing according to an exemplary embodiment of the present invention;

fig. 2 is a flowchart of a phase-unwrapped three-dimensional face reconstruction method for seed-level line pairing according to an exemplary embodiment of the present invention;

FIG. 3 is a schematic diagram of truncated phase and truncated phase level marks resolved from left and right phase shifted images in an exemplary embodiment of the invention;

FIG. 4 is a schematic diagram of a texture map, a face feature point extraction map and selected feature points obtained by left and right phase shift image analysis according to an exemplary embodiment of the invention;

FIG. 5 is a schematic diagram illustrating alignment of selected feature points in an exemplary embodiment of the present invention;

FIG. 6 is a line drawing on a human face and a schematic view of a truncated phase level set of lines defined by the intersection of the line drawing and the level line in an exemplary embodiment of the invention;

FIG. 7 is a schematic diagram of a truncated phase order sub-pair of indicia and a matched disparity matching matrix in an exemplary embodiment of the invention;

fig. 8 is a high-density disparity map and a three-dimensional face reconstruction result map obtained by the embodiment of the invention.

The labels in the figure are: 201-left camera, 202-right camera, 100-light field projection device, 400-control module.

Detailed Description

The present invention will be described in further detail with reference to test examples and specific embodiments. It should be understood that the scope of the above-described subject matter is not limited to the following examples, and any techniques implemented based on the disclosure of the present invention are within the scope of the present invention.

Example 1

A phase unwrapped three-dimensional face reconstruction system with a secondary line pairing, comprising: the system comprises a light field projection device, M cameras and a control module;

the control module is configured to send a control signal to a light field projection device so as to enable the light field projection device to project a light field sequence with a sine stripe structure and adjustable image quantity to the surface of a human face, and the three-dimensional human face modeling is completed through a phase-free unfolding three-dimensional human face reconstruction method based on human face images transmitted by a camera, wherein the phase-free unfolding three-dimensional human face reconstruction method is provided by the invention and is matched with a secondary line; the control module can be further used for adjusting the working frame frequency of the light field sequence of the sine stripe structure projected by the stroboscopic stripe structure light projection device and the working frame frequency of the image collected by the camera according to the working state of the system;

the light field projection device is used for projecting the sinusoidal stripe structure light field sequence with adjustable image quantity to the surface of a human face, and when receiving a control signal sent by the control module, the light field projection device performs N frames of stripe structure light projection and sends N synchronous control signals to the M cameras;

the M cameras are used for shooting multi-angle images of the face surface under the illumination of the fringe structure light field under the control of a synchronous control signal and transmitting the shot face images to the control module;

wherein N is an integer of 3 or more, and M is an integer of 2 or more.

The phase-unwrapped three-dimensional face reconstruction method for the secondary line pairing provided by the embodiment comprises the following steps:

s1, respectively collecting face images of N frames of detected faces in a structured light field at different M shooting angles; performing epipolar line correction on the acquired light field image of the structure; wherein N is an integer of 3 or more, and M is an integer of 2 or more;

s2, respectively resolving a truncation phase in the structure light field image after polar line correction to obtain a truncation phase diagram; obtaining a truncated phase level line by truncating the phase; analyzing texture images in the structured light field images after polar line correction, and extracting characteristic points; selecting and aligning the characteristic points to obtain a reference point group;

specifically, step S2 includes: s21, correspondingly selecting T reference points from the feature points extracted from the different texture images, wherein T is more than or equal to 2;

and S22, correspondingly moving the selected reference points to the secondary line and carrying out pairing calculation to obtain a reference point group.

The reference points include: human face characteristic points and/or human face characteristic point derivative points; the human face feature points include: eye, eyebrow, nose tip, corner of mouth, or cheek contour feature points; the face characteristic point derivative point is the middle point of two face characteristic points or the center of a plurality of face characteristic points.

In practical use, the following can be selected: the method comprises the following steps of moving the combination to a level secondary line, carrying out pairing calculation to obtain a reference point group, wherein the combination comprises a left pupil characteristic point, a right pupil characteristic point, a nose tip characteristic point, a plurality of eyebrow characteristic point centers, a plurality of left pupil characteristic point centers, a plurality of right pupil characteristic point centers, a plurality of cheek contour characteristic point centers and the like.

Further, in consideration of the fact that a slight pixel error may exist in the extraction of the same feature point in different pictures, the feature points need to be aligned, so that the feature points in different pictures after alignment are absolutely matched. In practical use, the feature point in one picture can be moved to the closest level line, and then the same feature point on other pictures can be moved to the corresponding level line in the picture in which the feature point is located by the stereo matching algorithm.

It should be noted that the stereo matching algorithm may be implemented by a zero mean cross correlation (ZNCC) algorithm, a Sum of Absolute Differences (SAD) algorithm of pixel gray, and the like, and the feature point extraction may be implemented by an algorithm such as Dlib, setaface, and the like.

S3, matching the truncated phase level secondary line through the reference point group to obtain a truncated phase level secondary line group; matching all pixel points in the same truncated phase level sub-line group by using a search algorithm and marking level serial numbers;

specifically, step S3 includes: s31, drawing lines on the face image according to a preset path by taking the reference point group as a reference, marking and sequencing intersection points of the drawn lines and the secondary lines, and arranging the serial numbers of the secondary lines according to the intersection points; placing the same serial number of the secondary lines into a truncated phase-level secondary line group;

and S32, marking the corresponding level serial number of each pixel point on each level line in the same truncated phase level line group through a search algorithm.

The preset path needs to pass through the points in the reference point group and needs to pass through all the secondary lines in the face image.

S4, obtaining a parallax matching matrix through matching of the truncated phase level secondary line groups;

specifically, in step S4, with a pixel on a secondary line in any one of the images as a reference, searching for a pixel on the same row with the same sequence number as the secondary line on the other secondary lines of the truncated phase secondary line group where the pixel is located, extracting a vertical coordinate of the pixel, and adding the vertical coordinate to the parallax matching matrix; if the same row does not search for the same order number, 0 is placed in the disparity matching matrix.

It should be noted that, elements in the parallax matching matrix are respectively composed of abscissa from a pixel in one picture and ordinate (or 0) from a pixel in another picture; therefore, by using the parallax matching matrix, a plurality of absolute matching points in different pictures can be accurately acquired, a dense parallax map is obtained based on the points, and three-dimensional reconstruction is carried out. In the existing three-dimensional reconstruction method without phase unwrapping, the number of points which are absolutely matched and are obtained by the method is small, or the obtained points do not guarantee the absolute matching; therefore, a sparse disparity map is often obtained, or a plurality of holes exist in the obtained disparity map, and the holes are filled by a hole filling algorithm; the method not only increases the time of the whole three-dimensional reconstruction, but also reduces the accuracy of the three-dimensional reconstruction.

And S5, matching the truncated phase images by using the parallax matching matrix to obtain a dense parallax image, and reconstructing a three-dimensional face according to the dense parallax image.

It should be noted that the truncated phase level line is a curve formed by the jumping pixel points in the truncated phase information; and the jumping pixel points are pixel points for jumping the phase value in the truncated phase diagram from pi to pi or from-pi to pi.

Example 2

Fig. 1 shows a schematic diagram of a three-dimensional face reconstruction system with no phase unwrapping and with a secondary line pairing proposed in this embodiment, and the system mainly consists of a light field projection apparatus 100, a left camera 201, a right camera 202 and a control module 400.

The control module is configured to send a control signal to a light field projection device, so that the light field projection device projects a light field sequence with a sine stripe structure and an adjustable image quantity to the surface of a human face, and three-dimensional human face modeling is completed based on a human face image transmitted by a camera; the control module is further used for adjusting the working frame frequency of the light field sequence of the sine stripe structure projected by the stroboscopic stripe structure light projection device and the working frame frequency of the image collected by the camera according to the working state of the system;

the light field projection device is used for projecting the sine stripe structure light field sequence with adjustable image quantity to the surface of a human face, and when receiving a control signal sent by the control module, the light field projection device performs 3-frame stripe structure light projection and sends 3 synchronous control signals to the left camera and the right camera;

the left camera and the right camera are both used for shooting multi-angle images of the face surface under the illumination of the fringe structure light field under the control of the synchronous control signal, and transmitting the shot images to the control module.

Fig. 2 shows a phase unwrapped three-dimensional face reconstruction method for a secondary line pairing, which includes:

step 501, the system provided by this embodiment is used to shoot a face image under illumination of a fringe structure light field, the left and right cameras respectively obtain 3 images, epipolar correction is performed according to system calibration information, and the truncated phase and truncated phase level mark lines (level lines) of the left and right cameras are analyzed. When the sine stripes are projected on the surface of the three-dimensional object, the 3 frames of deformation stripes are shot as follows:

wherein, I_bias(x, y) is the background intensity of the fringe image pixel (x, y), I_mod(x, y) is the modulation intensity, φ (x, y) is the desired truncation phase, and α is the phase shift step size 2 π/3. By the formulas (1) to (3), the truncation phase can be calculated by the following formula:

according to the jump edge from the truncation phase-pi to pi, the level marking line of the truncation phase can be analyzed, and the extraction results of the truncation phase and the level line are shown in fig. 3.

Step 502, respectively analyzing texture information contained in the fringe pattern after epipolar line correction to form a texture image pair, performing detection on the human face characteristic points, and selecting a left camera reference point pair and a right camera reference point pair. The texture map can be calculated by the following formula:

in the example, dilib is adopted to respectively detect the human face characteristic points on the texture image pair, and 68 characteristic points are detected in total; in order to increase the reliability of the setting reference, in this embodiment, 4 points are respectively selected from the left and right face images as reference points, which are respectively: the middle points of the left-eye feature points 34-40, the middle points of the right-eye feature points 43-46, the middle points of the left-eyebrow feature points 18-22, and the middle points of the right-eyebrow feature points 23-27. The texture map, the face feature point map, and the selected reference point map obtained are shown in fig. 4.

Step 503, performing left-right alignment on the selected reference points, firstly moving all the reference points to a nearby level, then calculating and marking left-right matching point pairs through a zero-mean normalized cross correlation (ZNCC) stereo matching algorithm to form left-right aligned reference point pairs, and comparing the reference point pairs before and after alignment as shown in fig. 5.

The ZNCC algorithm is a similarity stereo matching algorithm, a point pair with the maximum matching value S is used as a matching point pair, and the specific calculation formula is as follows:

wherein L (i + x, j + y) represents the pixel gray scale in the left image matching window matrix, and m_iRepresenting the average value of the gray levels of the pixels of the corresponding left image window; r (i + x, j + y) represents the pixel gray scale in the right image matching window matrix, m_tRepresenting the mean value of the grey levels, S, of the pixels corresponding to the right picture window_ZNCC(x, y) representsThe matching result, the result of which is in the range of 0 to 1, is higher the larger the value is, the higher the degree of similarity is.

S504, drawing lines on the left and right faces respectively according to a preset path, marking and sorting all points intersecting with the secondary line on the drawn line path, and then completing pairing of local points on the left and right secondary lines according to a positioning reference (4 pairs of reference points) and the intersection points, and the obtained drawing line on the face and the schematic diagram of pairing of local points on the left and right secondary lines are shown in fig. 6.

The method comprises the following specific steps:

a: predetermined drawing paths Path1 and Path2 are respectively set on the left half face and the right half face of the human face, and in the embodiment, the drawing paths and directions of the left half face and the right half face are respectively set as Path1 ═ 1,2,3,4,5,6,7, and Path2 ═ 1,8,9,10,11,12, 7.

Specifically, the Path1 ═ {2,3} and the Path2 ═ 8,9} in the Path are reference points that belong to the aligned and located on the secondary line obtained in step 504; the rest points are points expanded on the basis of the human face feature points, where Path1 ═ 4,5,6} is the midpoint of the connecting lines between feature points 3 and 32, 5 and 49, and 7 and 60, Path2 ═ 10,11,12} is the midpoint of the connecting lines between feature points 15 and 36, 13 and 55, and 11 and 56, Path1 ═ 7} and Path2 ═ 7} are the 9 th point of the feature point at the mandible, and Path1 ═ 1} and Path2 ═ 1} are the human face pixel points whose final value is-NaN is obtained by searching vertically upward from the 28 th point of the feature point toward the vertex (NaN represents no data, and-NaN represents data).

b: drawing lines along the set paths Path1 and Path2 from the top down direction, recording the truncation phase values line _ wrap (1 matrix M, M represents the number of all pixels passing through the drawn lines) corresponding to all the passing pixels on the drawn lines along the paths, then respectively calculating the expansion phase lines _ wrap on the two drawn lines according to the jump relation between the truncation phases-pi to pi, and recording the coordinates (x) of the intersection points of the two drawn lines and all the secondary lines (x is the intersection point of the two drawn lines)_m,y_m) And the serial number of the level line, m is the number of the level line on the face.Specifically, the order number of the rank line can be obtained from K ═ line _ unwrap/2 pi.

c: and taking 4 pairs of datum points as a positioning sequence, respectively adjusting the sequence mark consistency between 2 known-level sequence number points on each level line on the left face and the right face, and then taking the sequence mark consistency as a reference, and defining the level lines with the consistent sequence numbers of the known point marks on the faces corresponding to the left camera and the right camera as a pair of level lines.

Step 505, the rank serial number corresponding to each pixel point on each rank line is marked by a correlation search algorithm, and a full-pixel point matching pair of the left and right rank lines, i.e. a truncated phase rank mark line pair, is obtained, as shown in fig. 7, each pixel point of the paired rank lines in the left and right rank lines has the same rank serial number.

Specifically, the line search algorithm takes the known rank numbers a1 and a2 on one rank line as an example, and searches the line search windows Win1 up and down from points a1 and a2 respectively to a1 ± wndSize1 and Win1 to a2 ± wndSize 1. Each stripe projected on the face in this example occupies about 23 pixels, and typically, wndSize1 in this example may take 8.

And step 506, fitting a parallax matching matrix K according to the obtained truncated phase level secondary marker line pair for matching the parallax map, as shown in FIG. 7.

Specifically, K is based on the left camera truncated phase order line and is on the same line (x)_L＝x_R) The serial number of the rank line is a search signal, and the pixel coordinate (x) of the corresponding left rank line in the K matrix_L,y_L) Stores the row coordinate y of the point matched with the cut phase order line of the right camera_RI.e. K (x)_L,y_L)＝y_R(ii) a If no point matched with the abscissa of the right camera exists in the truncated phase level line, storing 0;

and 507, directly performing stereo matching on the basis of the truncated phase according to the fitted matching matrix M to obtain a high-precision dense disparity map of the sub-pixels of the face, as shown in FIG. 8.

Specifically, the search is performed on the matching matrix in rows: from M (x)_L,y_L)＝y_RIn a clear view of the above, it is known that,left camera truncates phase interval phi_L(x_L,y_L),φ_R(x_L+1,y_L+1)]Phase interval phi cut off from left camera_R(x_R,M(x_L,y_L)),φ_R(x_R+1,M(x_L+1,y_L+1)]And matching the interval for the correct parallax, and so on.

And step 508, calculating a three-dimensional model of the face to be detected according to the disparity map obtained by matching and the system calibration information.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.