阅读说明：本技术 一种正畸矫治过程中的牙齿矫治状态的检测方法 (Method for detecting tooth correction state in orthodontic correction process ) 是由 陈莉 金林荣 于 2021-01-27 设计创作，主要内容包括：本发明属于数字化医疗领域中的数字化口腔正畸技术领域,特别涉及一种正畸矫治过程中的牙齿矫治状态的检测方法。本发明利用拍摄的患者牙齿照片,基于计算机二维图像处理技术和计算机三维建模技术,来低成本地检测患者的牙齿矫治状态。本发明使得患者在家中进行牙齿矫治状态检测成为可能,无需患者去医院进行口腔扫描或者牙齿咬模,大大降低了患者和医院的成本。同时,本发明方案直接利用患者在进行正畸治疗前的牙齿模型来进行患者牙齿矫治状态检测,因此能够直接复用患者之前的数字化正畸方案,大大降低了正畸的成本。(The invention belongs to the technical field of digital orthodontic in the field of digital medical treatment, and particularly relates to a method for detecting a tooth correction state in an orthodontic correction process. The invention utilizes the taken tooth photo of the patient and detects the tooth correcting state of the patient at low cost based on the computer two-dimensional image processing technology and the computer three-dimensional modeling technology. The invention enables the patient to carry out tooth correction state detection at home, and the patient does not need to go to a hospital for oral cavity scanning or tooth biting, thereby greatly reducing the cost of the patient and the hospital. Meanwhile, the tooth model of the patient before orthodontic treatment is directly utilized to detect the tooth correcting state of the patient, so that the digital orthodontic scheme before the patient can be directly reused, and the orthodontic cost is greatly reduced.)

1. A method for detecting tooth correcting state in orthodontic correction process is characterized by comprising the following steps:

acquiring tooth correction state information;

preprocessing the digital oral cavity mesh model to obtain a digital oral cavity mesh model;

using a two-dimensional-three-dimensional registration method based on projection to carry out rigid transformation on the three-dimensional grid model of each tooth, projecting the three-dimensional grid model of each tooth onto a two-dimensional plane of the photo, and registering the two-dimensional projection tooth profile of each tooth with the tooth profile of the patient to obtain a real rigid transformation matrix of each tooth under the coordinate system of the photo;

acquiring an association matrix of two photo coordinate systems according to the real rigid transformation matrix of the teeth under each photo coordinate system;

and obtaining the whole set of tooth model according to the coordinate system correlation matrix and the rigid transformation matrix, and realizing the detection of the tooth correction state in the orthodontic correction process.

2. The detection method according to claim 1, wherein the obtaining of the orthodontic state information comprises the following steps:

(1) before invisible orthodontic treatment, scanning teeth of a patient to obtain a digital oral cavity mesh model of the patient, and storing position information of three-dimensional space points, lines and surfaces of the mesh model in the digital oral cavity mesh model;

(2) the method comprises the steps of taking a picture of the teeth of a patient with a label schematic diagram from different angles, wherein the picture of the teeth of the patient is a colorful M-N two-dimensional image, M is the width of the two-dimensional image, and N is the height of the two-dimensional image.

3. The detection method according to claim 2, wherein the digitized oral mesh model of step (1) is preprocessed to obtain the digitized oral mesh model by the following steps:

(1) the digital oral cavity mesh model is divided to obtain a three-dimensional mesh model of each tooth, and the three-dimensional mesh model of each tooth is numbered to obtain a dental crown mesh model A with the number j_j；

(2) Carrying out distortion removal treatment on the dental picture of the patient to obtain a distortion-removed two-dimensional image M x N of the oral cavity of the patient;

(3) and acquiring camera internal parameters and camera external parameters when the picture is taken.

4. The method of claim 1, wherein the extracting the profile of the patient's teeth from the two-dimensional M x N image of the patient's mouth after the distortion removal process comprises:

(1) carrying out contour edge detection on the M x N two-dimensional image of the oral cavity of the patient by adopting a canny edge detection method to obtain the M x N two-dimensional image with edge information;

(2) carrying out Garbor filtering processing on the M x N two-dimensional image of the oral cavity of the patient to obtain the M x N two-dimensional image with contour information;

(3) processing the M x N two-dimensional image of the oral cavity of the patient in the step (2) by using a generative edge learning method to obtain an M x N two-dimensional image only with tooth profile information;

(4) fusing the three M x N two-dimensional images obtained in the steps (1), (2) and (3) by adopting a contour fusion extraction method to obtain the M x N two-dimensional image with the patient tooth contour, wherein the contour fusion extraction method comprises the following steps:

(4-1) setting a threshold value of a pixel value in the image, carrying out binarization processing on the M x N two-dimensional image obtained after Garbor filtering in the step (2-4-2), and judging an edge of the pixel value higher than the threshold value in the two-dimensional image after the binarization processing as an edge contour;

(4-2) performing 'intersection' operation on the M x N two-dimensional images obtained in the step (4-1), the step (1) and the step (3), namely determining points which are simultaneously determined as edge contours in the step (4-1), the step (1) and the step (3) to be edge points of the tooth contour;

(5) numbering each tooth contour in the M x N two-dimensional image with the patient tooth contour to obtain B_ij，B_ijThe outline of tooth number j in the ith dental photograph is shown.

5. The method of claim 1, wherein the patient's tooth profile B is measured_ijStep-by-step digital three-dimensional model A of teeth_jPerforming two-dimensional and three-dimensional registration, namely using a projection-based two-dimensional and three-dimensional registration method to perform rigid transformation on the three-dimensional grid model of each tooth so that the three-dimensional grid model of each tooth is projected onto a two-dimensional plane, projecting to obtain an M x N two-dimensional projected tooth profile of each tooth, registering the M x N two-dimensional projected tooth profile of each tooth with the tooth profile of the patient in the step (2-4) to obtain a real rigid transformation matrix of each toothWherein i represents the ith picture, j represents the number of the tooth, and the real rigid transformation matrix of the tooth with the number of j in the ith picture is obtainedThe method comprises the following steps:

(5-1) Each tooth Profile B from the ith photograph_ijDividing the tooth into two tooth row profiles according to the upper jaw face or the lower jaw face where the tooth is located, and respectively connecting the upper jaw face tooth row profile or the lower jaw face tooth row profile with the digital three-dimensional tooth model A_jPerforming two-dimensional three-dimensional registration to obtain a rigid transformation matrix T 'of corresponding maxillofacial teeth in the ith dental picture'_i，T_iRepresents theoretical value of rigidity transformation matrix of jaw face teeth, T'_iIs T_iThe calculated value of (a); the step of the two-dimensional and three-dimensional registration of the whole dentition is represented by the following formula:

using the rigid transformation matrix T of the entire maxillofacial tooth in the ith dental photograph_iThe tooth digital three-dimensional model A obtained in the step (2-1)_jCarrying out rigidity change to obtain a three-dimensional tooth model with rigidity changedWherein N is_iThe number of tooth profiles marked on the maxillofacial surface or the submaxillofacial surface in the tooth photo i; the rigid transformation matrix T is composed of six parameters, T_x、t_y、t_zAlpha, beta, gamma, wherein t_x、t_y、t_zRepresents the amount of translation in space, alpha, beta, gamma represent the angle of rotation in space, where T and T_x、t_y、t_zThe relationship of α, β, γ is:

the tooth registration process for the entire dentition is as follows:

(5-1-1) use functionTo the obtained three-dimensional model of the tooth with changed rigidityProjecting the three-dimensional grid model to a two-dimensional plane to obtain a two-dimensional projection graphWherein the functionDetermining the parameters of the camera and the parameters of the camera;

(5-1-2) Using function C, for the obtained two-dimensional projection patternExtracting the tooth profile to obtain a projected tooth profile

(5-1-3) distance metric function F using two contours in two-dimensional plane^2DTo projection tooth profileAnd tooth profile B_ijPerforming distance measurement to obtain measurement distance

(5-1-4) traversing all teeth on a set of dentition, repeating the steps (5-1-1), (5-1-2) and (5-1-3) for each tooth to obtain the measured distances of all teeth, averaging the measured distances of all teeth to obtain the average value of the distance measurement mean value of the whole set of dentitionWherein N is_iThe number of tooth profiles marked on the maxillofacial surface or the submaxillofacial surface in the tooth photo i;

(5-1-5) continuously adjusting the rigid transformation matrix T of the dentition using an optimization method_iTo make the teeth digital three-dimensional model A_jThe measurement distance between the projected contour and the entire dentition contour is minimized, and when the measurement distance is minimized, the rigid transformation matrix T 'at that time is obtained'_i；

(5-2) rigidly transforming the matrix T 'from the entire tooth row'_iThe contour B of each tooth of the ith picture_ijAnd step digital three-dimensional model A of tooth_jFurther rigidity change is carried out to obtain a rigidity transformation matrix T 'of further rigidity transformation'_ijTwo-dimensional three-dimensional registration of individual teethThe steps are shown in the following formula:

wherein, T_ijRepresents the tooth numbered j in the photograph i, and rigid transformation T 'is performed on the entire tooth row'_iThe rigidity transformation is performed on the basis of (1). Obtaining a rigidity change matrix T 'when the projection of the single tooth and the real outline distance are minimum'_ij(ii) a The method comprises the following specific steps:

(5-2-1) entire tooth row rigid transformation matrix T 'obtained according to the step (5-1)'_iAnd the tooth digital three-dimensional model A of the ith picture number j_jObtaining a tooth three-dimensional model after further rigid transformation

(5-2-2) use functionTo the obtained three-dimensional model of the tooth with changed rigidityProjecting the three-dimensional grid model to a two-dimensional plane to obtain a two-dimensional projection graphWherein the functionDetermining the parameters of the camera and the parameters of the camera;

(5-2-3) Using function C, for the obtained two-dimensional projection patternExtracting the tooth profile to obtain a projected tooth profile

(5-2-4) evaluation function (for measuring distance) F using two contours in two-dimensional plane^2DTo projection tooth profileAnd tooth profile B_ijPerforming distance measurement to obtain measurement distance

(5-2-5) continuously adjusting the rigid transformation matrix T of the tooth using an optimization method_ijTo make the teeth digital three-dimensional model A_jThe measured distance between the projected contour and the entire tooth row contour is minimized, and when the measured distance is minimized, a rigid transformation matrix T 'of the tooth numbered j on the picture i is obtained'_ij；

(5-2-6) repeating the steps (5-2-1) to (5-2-5) for each tooth to obtain rigid transformation T 'of each tooth in the whole set of dentition'_iRigid transformation matrix T 'of further rigid transformation at top'_ij；

(5-3) the entire tooth row rigid transformation matrix T 'obtained in the step (5-1)'_iAnd the rigid transformation matrix T 'of each tooth obtained in the step (5-2) is further rigidly transformed'_ijObtaining the real rigid transformation matrix of the tooth numbered j in the ith picture

6. The method of claim 1, wherein the detection is based on a true rigid transformation matrix of the tooth numbered j in the ith pictureObtaining a photograph i of a tooth having n identical numbers₁And a photograph i₂Coordinate system correlation matrix ofThe method comprises the following steps:

(6-1) taking photograph i₁And a photograph i₂N teeth with the same number are marked as n common teeth;

(6-2) determining a correlation matrix in the coordinate system with respect to the tooth with the number k as a reference, by taking the tooth with the number k as an example, k is not equal to i, and k is not equal to j, in the n teeth with the common numberIn association, a photo i is obtained₁And a photograph i₂The projected metric distance D of n common teeth_kWherein, in the step (A),represents from i₁Conversion to i₂A coordinate system correlation matrix of a coordinate system, to representInverse matrix of, D_kThe calculation formula of (a) is as follows:

using the tooth with the number k as a reference, in the photograph i₁In the coordinate system of (1), the obtained image is obtained in the picture i₂I to the picture of n common teeth₁Projective metric distance of imaging platformObtaining a projection metric distanceThe steps are as follows:

(6-2-1) according to photograph i₂True rigid transformation matrix for the middle numbered j toothAnd a three-dimensional tooth mesh model after rigid transformation

(6-2-2) according to photograph i₁And a photograph i₂Coordinate system relation matrix ofPhotograph i obtained from step (6-1-1)₂Three-dimensional mesh model of middle numbered j teethGet a transfer to photograph i₁Three-dimensional mesh model of teeth

(6-2-3) use functionFor the three-dimensional network model of the teeth after the rigidity change in the step (6-1-2)Projecting the three-dimensional grid model to a two-dimensional plane to obtain a two-dimensional projection graph

(6-2-4) Using function C, on the obtainedTwo-dimensional projection patternExtracting the tooth profile to obtain a projected tooth profile

(6-2-5) distance metric function F using two contours in two-dimensional plane^2DTo projection tooth profileAnd tooth profile B_ijPerforming distance measurement to obtain measurement distance

(6-2-6) photograph i of the tooth with the number k as a reference₁In the coordinate system of (1), the obtained image is obtained in the picture i₂I to the picture of n common teeth₁Projection average metric distance of imaging platformNamely, it is

(6-2-7) repeating the steps (6-1-2) to (6-1-6), and obtaining the tooth k in the photograph i based on the number k₂In the coordinate system of (1), in the photograph i₁Projection of n commonly numbered teeth in (a) onto photograph i₁Projection average metric distance of imaging platformNamely, it is

(6-2-8) projection average metric distanceAnd projection average metric distanceAdding the numbers to obtain a picture i based on the number k of teeth₁And a photograph i₂The projection metric distance D of the N common teeth_k；

(6-2-9) repeating the steps (6-2-1) to (6-2-8) to obtain projection measurement distances of the n common teeth by taking each tooth as a reference;

and (6-3) obtaining the projection measurement distance based on each tooth according to the step (6-2) in the n common teeth. Finding out the tooth number with the minimum measuring distance from the measuring distances of all teeth, marking as the tooth number k, and using a coordinate system correlation matrix of the tooth with the number kAs a photograph i₁And a photograph i₂Coordinate system correlation matrix of

(7) Photograph i of tooth j according to the same number₁And a photograph i₂Coordinate system correlation matrix ofAnd the rigid transformation matrix of the tooth numbered j on the photograph iAnd obtaining the whole pair of tooth models, and realizing the detection of the tooth correction state in the orthodontic correction process.

Technical Field

The invention belongs to the technical field of digital orthodontic in the field of digital medical treatment, and particularly relates to a method for detecting a tooth correction state in an orthodontic correction process.

Background

Orthodontics, namely teeth correction, is realized by using external force to pull teeth through correcting a bracket, a correcting tooth socket and the like, so that the teeth are arranged neatly. The digital orthodontic treatment is to apply a computer-aided technology to orthodontic treatment, and carry out the steps of digital tooth data acquisition, storage, orthodontic scheme design and the like.

In the process of orthodontic treatment, a patient often needs to go to a hospital for a double-check to check the orthodontic state of teeth. The existing tooth correction state detection methods mainly comprise 3 methods: the first method is that a patient goes to a hospital, a doctor inspects the teeth of the patient on site, and the tooth correction state of the patient is detected; the second method is that the patient goes to a hospital, a plaster model is used for biting the teeth of the patient, or a professional oral scanner is used for carrying out three-dimensional modeling on the teeth of the patient, so as to obtain a tooth three-dimensional scanning model of the patient; the third is that the patient takes a picture of his own teeth with a mobile phone, camera, etc., and then sends the picture to the doctor.

The method has the disadvantages that 1, if only the teeth of a patient are photographed, only two-dimensional information of the teeth can be obtained, but three-dimensional information of the patient cannot be obtained, and the three-dimensional tooth correction state of the patient cannot be well quantized; 2. if the plaster occlusion model is used for taking a model, a patient needs to go to a hospital for operation, and the cost is relatively high for both the hospital and the patient; 3. the professional oral cavity scanner is used for scanning the oral cavity model, a patient needs to go to a hospital for operation, and the cost is relatively high for both the hospital and the patient; and the oral cavity scanning model is obtained by scanning, the steps of dental crown segmentation, orthodontic scheme design and the like need to be carried out again, the original digital orthodontic scheme cannot be directly used, and time and labor are wasted. 4. The doctor detects the patient's tooth at the scene, and the state inspection is corrected and is relied on doctor's experience to the patient, can't carry out the quantitative measurement, and the inspection result also can't be digitalized, can't reuse original digital orthodontic design scheme.

Disclosure of Invention

The invention aims to provide a tooth correction state detection method in an orthodontic correction process, which is used for detecting the tooth correction state of a patient at low cost by utilizing a shot tooth photo of the patient and based on a computer two-dimensional image processing technology and a computer three-dimensional modeling technology.

The invention provides a method for detecting tooth correction state in an orthodontic correction process, which comprises the following steps:

acquiring tooth correction state information;

preprocessing the digital oral cavity mesh model to obtain a digital oral cavity mesh model;

using a two-dimensional-three-dimensional registration method based on projection to carry out rigid transformation on the three-dimensional grid model of each tooth, projecting the three-dimensional grid model of each tooth onto a two-dimensional plane of the photo, and registering the two-dimensional projection tooth profile of each tooth with the tooth profile of the patient to obtain a real rigid transformation matrix of each tooth under the coordinate system of the photo;

acquiring an association matrix of the coordinate systems of the two photos according to the real rigid transformation matrix of the teeth under the coordinate system of each photo;

and obtaining the whole set of tooth model according to the coordinate system correlation matrix and the rigid transformation matrix, and realizing the detection of the tooth correction state in the orthodontic correction process.

The method for detecting the tooth correcting state in the orthodontic correcting process has the advantages that:

the method for detecting the tooth correcting state in the orthodontic correction process provided by the invention is used for detecting the tooth correcting state of a patient at low cost by utilizing a shot tooth photo of the patient and based on a computer two-dimensional image processing technology and a computer three-dimensional modeling technology. The method of the invention enables the patient to carry out tooth correction state detection at home, and the patient does not need to go to a hospital for oral cavity scanning or tooth biting, thereby greatly reducing the cost of the patient and the hospital. Meanwhile, the tooth model of the patient before orthodontic treatment is directly utilized to detect the tooth correcting state of the patient, so that the digital orthodontic scheme before the patient can be directly reused, and the orthodontic cost is greatly reduced.

Drawings

Fig. 1 is a flow chart of a method for detecting a tooth correction state in an orthodontic correction process according to the present invention.

Detailed Description

The invention provides a method for detecting tooth correction state in an orthodontic correction process, which comprises the following steps:

acquiring tooth correction state information;

preprocessing the digital oral cavity mesh model to obtain a digital oral cavity mesh model;

using a two-dimensional-three-dimensional registration method based on projection to carry out rigid transformation on the three-dimensional grid model of each tooth, projecting the three-dimensional grid model of each tooth onto a two-dimensional plane of the photo, and registering the two-dimensional projection tooth profile of each tooth with the tooth profile of the patient to obtain a real rigid transformation matrix of each tooth under the coordinate system of the photo;

acquiring an association matrix of the coordinate systems of the two photos according to the real rigid transformation matrix of the teeth under the coordinate system of each photo;

and obtaining the whole set of tooth model according to the coordinate system correlation matrix and the rigid transformation matrix, and realizing the detection of the tooth correction state in the orthodontic correction process.

The invention is described in detail below with reference to the accompanying drawings:

the flow chart of the method for detecting the tooth correcting state in the orthodontic correcting process is shown in figure 1, and the method comprises the following steps:

(1) acquiring tooth correction state information, and the specific process is as follows:

(1-1) before invisible orthodontic treatment, scanning teeth of a patient by using a three-dimensional scanning gun to obtain a digital oral cavity mesh model of the patient, wherein the digital oral cavity mesh model is stored in an obj (3D model file format) file format commonly used in the industry, and position information of three-dimensional space points, lines and planes of the mesh model is stored in the digital oral cavity mesh model;

(1-2) taking a photo of the teeth of a patient with a label schematic diagram from different angles, taking the photo of the teeth of the patient by using a mobile phone, wherein the photo of the teeth of the patient is a colorful M x N two-dimensional image, the label schematic diagram is a black and white square image similar to a two-dimensional code, M is the width of the two-dimensional image, and N is the height of the two-dimensional image;

(2) preprocessing the digital oral mesh model in the step (1), wherein the specific process is as follows:

and (2-1) carrying out tooth distribution operation on the intraoral scanning grid model of the patient before the invisible orthodontics. The tooth distribution operation comprises the steps of dividing the digital oral cavity mesh model in the step (1-1) to obtain a three-dimensional mesh model of each tooth, numbering the real teeth according to orthodontics, numbering the three-dimensional mesh model of each tooth to obtain a dental crown mesh model A with the number j_j；

The camera intrinsic parameters for taking a picture are acquired (the camera intrinsic parameters are parameters related to the characteristics of the camera itself, such as the focal length of the camera, the pixel size, and the like).

(2-2) carrying out distortion removal treatment on the dental picture of the patient in the step (1-2) to obtain a distortion-removed two-dimensional image M x N of the oral cavity of the patient; each patient picture taken is de-distorted using the image processing tool OpenCV (a cross-platform computer vision and machine learning software library based on BSD licensing).

(2-3) acquiring camera internal parameters and camera external parameters when the picture is taken in the step (1-2); (an out-of-camera parameter is a parameter in a world coordinate system such as the position, rotational direction, etc. of the camera). When a picture of the patient's teeth is taken, a label of a custom label (1cm x 1cm, similar to a two-dimensional code) is applied to the patient's lips so that the label can appear in the picture. The extrinsic parameters of the camera are obtained from the position of the tag in the picture, a method that is known and used in common.

(2-4) extracting the tooth profile of the patient from the two-dimensional image of the patient's oral cavity M x N after the distortion removal treatment in the step (2-2), wherein the specific process is as follows:

(2-4-1) carrying out contour edge detection on the M x N two-dimensional image of the oral cavity of the patient in the step (2-2) by adopting a Canny edge detection method to obtain an M x N two-dimensional image with edge information;

in the low-cost tooth correction state detection method provided by the invention, the contour in a tooth photo taken by a patient is mainly used as two-dimensional information to carry out two-dimensional and three-dimensional registration. Contour extraction of a dental picture submitted by a patient is therefore required.

Regarding the processing of tooth contour in photo, the present invention uses various picture contour extraction methods to extract the contour of tooth in photo, and then fuses the results of these methods to obtain the contour of tooth (tooth contour refers to the contour at crown, excluding the contour at the boundary line between tooth and gum)

The light in the oral cavity is relatively insufficient, the photographing equipment of a patient is generally equipment such as a mobile phone, a plurality of noise points are easy to appear in a photographed dental picture, and the Canny algorithm is not easy to be influenced by the noise points in the picture and can accurately extract edges, so that the Canny algorithm is used for extracting the contour edges in the picture.

(2-4-2) carrying out Garbor filtering processing on the M x N two-dimensional image of the oral cavity of the patient in the step (2-2) to obtain an M x N two-dimensional image with contour information;

the tooth profile shape is special (mostly consisting of a horizontal or vertical arc), and because the light changes greatly when the patient takes a picture of the teeth in the mouth (e.g., no flash, indoor or outdoor taking, whether the light source is blocked, etc.), the extraction of the tooth profile of the patient needs to take into account the light during taking the picture. Since Garbor filtering is not sensitive to illumination and can extract information difficult to extract in a spatial domain from a frequency domain, a direction in which a contour is extracted can be selected. Therefore, the invention uses Garbor filtering to carry out edge contour extraction on the pictures submitted by the patients from four directions of 0 degree, 45 degrees, 90 degrees and 135 degrees.

(2-4-3) processing the M x N two-dimensional image of the oral cavity of the patient in the step (2-2) by using a generating Edge Learning method (Boost Edge Learning (BEL) for short) to obtain an M x N two-dimensional image only with tooth profile information;

because Canny algorithm and Garbor filtering extract the global edge contour in the picture, the invention needs to extract the contour of the patient's teeth (excluding the edges of the gums and teeth). There is a need for an edge contour extraction algorithm to extract the profile of a patient's teeth. Because the present invention has a relatively small amount of data for dental profile training data, the present invention learns the dental profile of a patient with a small number of training samples using the BEL method with probabilistic boosting trees.

The BEL algorithm, when implemented in detail, determines independently for each point in the image whether it is a contour point. In the judgment, a very large aperture (49 × 49 in the present invention) was used as a background for judgment with this point as the center. In the learning phase, the BEL algorithm selects and combines a large number of features at different scales in order to learn the discriminative model using an extended version of the probabilistic boosting tree classification algorithm.

(2-4-4) fusing the three M X N two-dimensional images obtained in the step (2-4-1), the step (2-4-2) and the step (2-4-3) by adopting a contour fusion extraction method to obtain the M X N two-dimensional image with the contour of the teeth of the patient, and comprising the following steps of:

the Canny algorithm has good insensitivity to noise and can carry out accurate edge extraction, but can extract a global edge profile; the Garbor filtering is not sensitive to the illumination of the patient when the patient shoots the teeth, the direction of the contour is well selected, and the edge contour can be extracted; the BEL algorithm only extracts the contours of the patient's teeth, but accuracy is to be improved and is relatively sensitive to noise. Through the fusion of the three methods, advantages are made good for and disadvantages are avoided, the advantages of each method are obtained, and the accurate tooth profile can be obtained.

(2-4-4-1) setting a threshold value of a pixel value in the image, carrying out binarization processing on the M x N two-dimensional image obtained after Garbor filtering in the step (2-4-2), and judging an edge of which the pixel value is higher than the threshold value in the two-dimensional image after the binarization processing as an edge contour;

(2-4-4-2) performing 'intersection' operation on the M x N two-dimensional images obtained in the step (2-4-4-1), the step (2-4-1) and the step (2-4-3), namely determining points which are simultaneously judged to be edge contours in the step (2-4-1), the step (2-4-1) and the step (2-4-3) to be edge points of the tooth contour;

(2-5) numbering each tooth profile in the M x N two-dimensional image with the patient tooth profile obtained in the step (2-4), namely obtaining B_ij，B_ijRepresenting the outline of the tooth with the number j in the ith tooth photo;

since the tooth shapes of each person are different and the tooth profiles obtained by photographing the teeth from different angles are also different, it is relatively difficult to number the obtained tooth profiles through machine learning. Moreover, the aforementioned contour extraction and fusion algorithm cannot achieve 100% accuracy, so that the tooth profile obtained by the aforementioned contour extraction and fusion algorithm needs to be simply modified manually and numbered according to the dentition table.

In order to ensure the accuracy of the next two-dimensional three-dimensional registration, when the tooth profiles are numbered manually, the fuzzy tooth profiles can be discarded, and the clear tooth profiles can be reserved. Although the step still needs manual interaction, compared with the manual contour extraction, the method greatly simplifies the complexity of the manual interaction, and the manual operation only needs to carry out numbering and simple modification on the basis of a contour extraction fusion algorithm.

(3) The tooth profile B of the patient in the step (2-5)_ijAnd (2) the digital three-dimensional model A of the tooth in the step (1)_jCarrying out registration (two-dimensional three-dimensional registration for short), namely, using a projection-based two-dimensional-three-dimensional registration method to carry out rigid transformation on the three-dimensional grid model of each tooth in the step (2-1) so that the three-dimensional grid model of each tooth is projected onto a two-dimensional plane, projecting to obtain an M x N two-dimensional projection tooth outline of each tooth (the tooth outline does not comprise a tooth gum intersection line), and registering the M x N two-dimensional projection tooth outline of each tooth with the tooth outline of the patient in the step (2-4) to obtain a real rigid transformation matrix of each toothWherein i represents the ith picture in the step (2-4), j represents the number of the tooth in the step (2-4), and the real rigid transformation matrix of the tooth with the number of j in the ith picture is obtainedThe method comprises the following steps:

(3-1) respectively carrying out two-dimensional three-dimensional registration on the tooth profile of each picture by taking the maxillofacial dentition and the mandible facial dentition as a whole, namely obtaining each tooth profile B of the ith picture according to the step (2-5)_ijDividing the tooth into two dentofacial profiles according to the maxillofacial or mandibular facial surfaces where the teeth are located, and respectively matching the maxillofacial or mandibular facial profile with the tooth digital three-dimensional model A obtained in the step (2-1)_jPerforming two-dimensional three-dimensional registration to obtain a rigid transformation matrix T 'of corresponding maxillofacial teeth in the ith dental picture'_i，T_iRepresents theoretical value of rigidity transformation matrix of jaw face teeth, T'_iIs T_iThe calculated value of (a); wherein the maxillary and mandibular dentition treatment steps are completely identical. The step of the two-dimensional and three-dimensional registration of the whole dentition is represented by the following formula:

using the rigid transformation matrix T of the entire maxillofacial tooth in the ith dental photograph_iThe tooth digital three-dimensional model A obtained in the step (2-1)_jCarrying out rigidity change to obtain a three-dimensional tooth model with rigidity changedWherein N is_iThe number of tooth profiles marked on the maxillofacial surface or the submaxillofacial surface in the tooth photo i; the rigid transformation matrix T is composed of six parameters, T_x、t_y、t_zAlpha, beta, gamma, wherein t_x、t_y、t_zRepresents the amount of translation in space, alpha, beta, gamma represent the angle of rotation in space, where T and T_x、t_y、t_zThe relationship of α, β, γ is:

the tooth registration process for the entire dentition is as follows:

(3-1-1) use functionTo the obtained three-dimensional model of the tooth with changed rigidityProjecting the three-dimensional grid model to a two-dimensional plane to obtain a two-dimensional projection graphWherein the functionDetermining the camera internal parameters and the camera external parameters obtained in the step (2-3); in the invention, an open graphics library OpenGL is used for projection, and a three-dimensional grid model is projected to a two-dimensional imaging plane through camera internal parameters and camera external parameters, and the method is a known technology in the technical field.

(3-1-2) Using function C, for the obtained two-dimensional projection patternExtracting the tooth profile to obtain a projected tooth profileIn the invention, the two-dimensional graph obtained by projection in the step (3-1-1) is binarized (namely, the background is black, and the projection of the tooth is white), and then a graph processing software OpenCV is used for projecting the tooth profileThe contour extraction method is also known in the art.

(3-1-3) distance metric function F using two contours in two-dimensional plane^2DTo projection tooth profileAnd the step (A) of2-5) derived tooth profile B_ijPerforming distance measurement to obtain measurement distance

In the implementation of the present invention, the average distance (euclidean distance) of the nearest pair is taken as the distance between the two contours. By { x^projectDenotes a set of contour points, K, generated by projection of a certain tooth^projectThe number of projected contour points; by { x^realDenotes a set of points corresponding to the true contour of the tooth in the photograph, K^realRepresenting the number of true contour points; the distance metric function F^2DCan be represented by the following formula:

wherein the content of the first and second substances,is a true contour pointThe nearest corresponding point in the set of projected contour points.

(3-1-4) traversing all teeth on a pair of dentitions, repeating the steps (3-1-1), (3-1-2) and (3-1-3) for each tooth to obtain the measured distances of all teeth, averaging the measured distances of all teeth to obtain the distance measured mean value of the whole pair of dentitionsWherein N is_iThe number of tooth profiles marked on the maxillofacial surface or the submaxillofacial surface in the tooth photo i;

(3-1-5) continuously adjusting the rigid transformation matrix T of the dentition using an optimization method_iTo make the teeth digital three-dimensional model A_jThe measured distance between the projected contour of (2) and the entire dentition contour obtained in step (2-5) is minimized, and when the measured distance is minimized, the rigid transformation matrix T 'at that time is obtained'_i；

(3-2) entire tooth row rigid transformation matrix T 'according to step (3-1)'_iAnd performing two-dimensional and three-dimensional registration on each tooth. That is, the entire set of tooth row rigid transformation matrices T 'according to step (3-1)'_iObtaining each tooth profile B of the ith picture from the step (2-5)_ijAnd step (2-1) of the digital three-dimensional model A of the tooth_jFurther rigidity change is performed to obtain a rigidity conversion matrix T 'of further rigidity conversion'_ijThe two-dimensional three-dimensional registration of a single tooth is shown as the following formula:

wherein, T_ijRepresents the tooth numbered j in the photograph i, and rigid transformation T 'is performed on the entire tooth row'_iThe rigidity transformation is performed on the basis of (1). Obtaining a rigidity change matrix T 'when the projection of the single tooth and the real outline distance are minimum'_ij(ii) a The method comprises the following specific steps:

(3-2-1) entire tooth row rigid transformation matrix T 'obtained according to the step (3-1)'_iAnd the tooth digital three-dimensional model A of the ith picture number j_jObtaining a tooth three-dimensional model after further rigid transformation

(3-2-2) use functionTo the obtained three-dimensional model of the tooth with changed rigidityProjecting the three-dimensional grid model to a two-dimensional plane to obtain a two-dimensional projection graphWherein the functionDetermining the camera internal parameters and the camera external parameters obtained in the step (2-3);

(3-2-3) Using function C, for the obtained two-dimensional projection PatternExtracting the tooth profile to obtain a projected tooth profile

(3-2-4) evaluation function (for measuring distance) F using two contours in two-dimensional plane^2DTo projection tooth profileAnd the tooth profile B obtained in the step (2-5)_ijPerforming distance measurement to obtain measurement distance

(3-2-5) continuously adjusting the rigid transformation matrix T of the tooth using an optimization method_ijTo make the teeth digital three-dimensional model A_jThe measured distance between the projection outline and the whole dentition outline obtained in the step (2-5) is minimum. When the metric distance is minimum, obtaining a rigid transformation matrix T 'of the tooth numbered j on the picture i'_ij；

(3-2-6) for each tooth, the operations (3-2-1), (3-2-2), (3-2-3), (3-2-4) and (3-2-5) were carried out to obtain T 'for the entire set of teeth rigid transformation of each tooth in step (3-1)'_iRigid transformation matrix T 'further rigidly transformed on basis'_ij；

(3-3) the entire tooth row rigid transformation matrix T 'obtained in the step (3-1)'_iAnd the rigid transformation matrix T 'of each tooth obtained in the step (3-2) is further rigidly transformed'_ijObtaining the real rigid transformation matrix of the tooth numbered j in the ith picture

(4) Obtaining the real rigid transformation matrix of the tooth numbered j in the ith picture according to the step (3)Obtaining a photograph i of a tooth having n identical numbers₁And a photograph i₂Coordinate system correlation matrix ofThe method comprises the following steps:

(4-1) taking photograph i₁And a photograph i₂N teeth with the same number are marked as n common teeth; the tooth number is a number which is universal for stomatology;

(4-2) determining a correlation matrix in the coordinate system with the number k as a reference, using the number k teeth as an example, k is not equal to i, and k is not equal to j, among the n common-numbered teethIn association, a photo i is obtained₁And a photograph i₂The projected metric distance D of n common teeth_kWherein, in the step (A),represents from i₁Conversion to i₂A coordinate system correlation matrix of a coordinate system, to representInverse matrix of, D_kThe calculation formula of (a) is as follows:

using the tooth with the number k as a reference, in the photograph i₁In the coordinate system of (1), the obtained image is obtained in the picture i₂I to the picture of n common teeth₁Projective metric distance of imaging platformObtaining a projection metric distanceThe steps are as follows:

(4-2-1) according to the step (3-3), photograph i is obtained₂True rigid transformation matrix for the middle numbered j toothAnd a three-dimensional tooth mesh model after rigid transformation

(4-2-2) according to photograph i₁And a photograph i₂Coordinate system relation matrix ofPhotograph i obtained from step (4-1-1)₂Three-dimensional mesh model of middle numbered j teethGet a transfer to photograph i₁Three-dimensional mesh model of teeth

(4-2-3) use functionFor the three-dimensional network model of the teeth after the rigidity change in the step (4-1-2)Projecting from a three-dimensional mesh model to a two-dimensional planeTo obtain a two-dimensional projection pattern

(4-2-4) Using function C, for the obtained two-dimensional projection PatternExtracting the tooth profile to obtain a projected tooth profile

(4-2-5) distance metric function F using two contours in two-dimensional plane^2DTo projection tooth profileAnd the tooth profile B obtained in the step (2-5)_ijPerforming distance measurement to obtain measurement distance

(4-2-6) photograph i of the tooth with the number k as a reference₁In the coordinate system of (1), the obtained image is obtained in the picture i₂I to the picture of n common teeth₁Projection average metric distance of imaging platformNamely, it is

(4-2-7) repeating the steps (4-1-2), (4-1-3), (4-1-4), (4-1-5) and (4-1-6) to obtain a photograph i of the tooth k of the reference number₂In the coordinate system of (1), the obtained image is obtained in the picture i₁Projection of n commonly numbered teeth in (a) onto photograph i₁Projection average metric distance of imaging platformNamely, it is

(4-2-8) projection average metric distanceAnd projection average metric distanceAdding the numbers to obtain a picture i based on the number k of teeth₁And a photograph i₂The projection metric distance D of the N common teeth_k；

(4-2-9) repeating the steps (4-2-1) to (4-2-8) to obtain the projection measurement distance of each tooth as a reference in the n common teeth.

And (4-3) obtaining the projection measurement distance based on each tooth according to the step (4-2) in the n common teeth. Finding out the tooth number with the minimum measuring distance from the measuring distances of all teeth, marking as the tooth number k, and using the coordinate system correlation matrix (the tooth with the number k) of the tooth number kAs a photograph i₁And a photograph i₂Coordinate system correlation matrix of

(5) Photograph i of identically numbered tooth j according to step (4)₁And a photograph i₂Coordinate system correlation matrix ofAnd the rigid transformation matrix of the tooth numbered j on the photo i obtained in the step (3)And obtaining the whole pair of tooth models, and realizing the detection of the tooth correction state in the orthodontic correction process.

After the tooth model is generated by the method, the tooth correcting state can be detected by using some metering modes in a digital orthodontic system. The specific measurement of the tooth correction state is different according to the requirements of different orthodontic companies, and each company can detect the tooth correction state of the generated model according to the requirements by using state measurement indexes such as PAR index, adjacent tooth space and the like.