阅读说明：本技术 一种耳显微外科术中关键指标实时三维测量的系统及方法 (System and method for real-time three-dimensional measurement of key indexes in ear microsurgery ) 是由 戴朴 张红蕾 邵航 刘威 于 2021-09-06 设计创作，主要内容包括：本发明提出了一种耳显微外科术中关键指标实时三维测量的系统及方法,涉及计算显微成像技术领域,包括：双视点显微成像单元,用于产生待测量显微手术中中耳及内耳结构对称的显微图像数据；双视点显微图像采集单元,用于采集所述显微图像数据；双视点显微图像控制与处理单元,用于负责控制所述双视点显微图像采集单元的拍摄,并将采集得到的显微图像数据进行处理以得到待测量指标的量化结果；所述双视点显微图像采集单元与所述双视点显微成像单元连接,所述双视点显微图像采集单元与所述双视点显微图像控制与处理单元连接；本发明通过显微成像技术进行精准的智能分析处理,并通过非接触性测量即可测得耳显微外科术野中关键的复杂指标。(The invention provides a system and a method for real-time three-dimensional measurement of key indexes in ear microsurgery, which relate to the technical field of computational microscopy imaging and comprise the following steps: the double-viewpoint microscopic imaging unit is used for generating microscopic image data of the symmetrical structure of the middle ear and the inner ear in the microscopic operation to be measured; the double-viewpoint microscopic image acquisition unit is used for acquiring the microscopic image data; the double-viewpoint microscopic image control and processing unit is used for controlling the shooting of the double-viewpoint microscopic image acquisition unit and processing acquired microscopic image data to obtain a quantitative result of an index to be measured; the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic imaging unit and is connected with the double-viewpoint microscopic image control and processing unit; the invention carries out accurate intelligent analysis and processing by microscopic imaging technology and can measure key complex indexes in the ear microsurgery field by non-contact measurement.)

1. A system for real-time three-dimensional measurement of key indicators in ear microsurgery, comprising:

the double-viewpoint microscopic imaging unit is used for generating microscopic image data of the symmetrical structure of the middle ear and the inner ear in the microscopic operation to be measured;

the double-viewpoint microscopic image acquisition unit is used for acquiring the microscopic image data;

the double-viewpoint microscopic image control and processing unit is used for controlling the shooting of the double-viewpoint microscopic image acquisition unit and processing the acquired microscopic image data to obtain a quantitative result of the index to be measured;

the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic imaging unit and is also connected with the double-viewpoint microscopic image control and processing unit.

2. The system of claim 1, wherein the dual viewpoint microscopic imaging unit comprises a light source, a large objective lens and a dual optical path continuous zoom body, the large objective lens is connected with the light source, the light source is connected with the dual optical path continuous zoom body, and the dual optical path continuous zoom body is connected with the dual viewpoint microscopic image acquisition unit; the light source is used for illuminating the surgical field, the large objective lens is used for adjusting the imaging focal plane of the microscopic imaging device, and the double-optical-path continuous zoom lens is used for changing the distance between lenses in the optical lens group.

3. The system of claim 1, wherein the dual-viewpoint microscopic image acquisition unit comprises a first visible light sensor and a second visible light sensor, the first visible light sensor provides the dual-viewpoint microscopic image acquisition unit with a first viewing angle as a first observation optical path for middle ear and inner ear structures in the microsurgical operation, the second visible light sensor provides the dual-viewpoint microscopic image acquisition unit with a second viewing angle as a second observation optical path for middle ear and inner ear structures in the microsurgical operation, and the first observation optical path and the second observation optical path are symmetrical.

4. The system of claim 3, wherein the first visible light sensor and the second visible light sensor both receive light emitted from the surface of the surgical field to be measured and respectively present an image of the middle ear and inner ear structures at the first viewing angle and an image at the second viewing angle during the microsurgery to be measured.

5. The system of claim 3, wherein the dual-viewpoint microscopic image control and processing unit comprises a synchronous trigger device, a dual-channel microscopic image acquisition card and a computer, the dual-viewpoint microscopic image acquisition unit is connected with the synchronous trigger device, the synchronous trigger device is connected with the dual-channel microscopic image acquisition card, and the dual-channel microscopic image acquisition card is connected with the computer; the synchronous triggering device is used for controlling the first visible light sensor and the second visible light sensor to synchronously shoot, the dual-channel microscopic image acquisition card is used for outputting data of the first visible light sensor and the second visible light sensor to the computer from a sensor end, and the computer is used for processing the dual-viewpoint microscopic image and calculating quantitative results of key indexes in structures of middle ear and inner ear in the microsurgery.

6. A method for real-time three-dimensional measurement of key indexes in ear microsurgery is characterized by comprising the following steps:

step S1, calibrating two visible light sensors in the double-viewpoint microscopic image acquisition unit, and acquiring internal parameters of the first visible light sensor and the second visible light sensor under each measurement magnificationAnd an external parameter T of the first visible light sensor relative to the second visible light sensor_i ¹²(ii) a Wherein subscript i is a particular microscopic imaging magnification;

step S2, adjusting the double-light-path continuous zoom body under a certain multiplying power i, and acquiring images of the middle ear and the inner ear structure parts in the operation field at the first visible light sensor by controlling the synchronous trigger deviceAnd images in the second visible light sensor

Step S3, based on internal and external parameters, using computer to look atStereo correction algorithm in vision Correcting to obtain a corrected image of the first visible light sensorAnd a corrected image of the second visible light sensorAnd a corrected reprojection matrix Q of the first visible light sensor;

step S4, correcting the imagePerforming a dense matching algorithm to obtain a corrected imageOf (d) a parallax map₁₂；

Step S5, correcting imageAndreprojection matrix Q and disparity map d₁₂Obtaining corrected images using triangulation methods in computer visionThe space coordinates of each point in the model are under the optical center coordinate system of the first visible light sensor, and all the points form a point cloud model of the structures of the middle ear and the inner ear in the microsurgery in the operation field;

step S6, after the point cloud model is obtained, the image is correctedObtaining a two-dimensional image point set L { L } of the facial nerve outline by man-machine interactive circle selection or automatic identification model identification_i(x_i,y_i) Y and a two-dimensional set of points at the outer edge of the crypt window_j(x_j,y_j) And a set of two-dimensional points Q Q of the two curved surfaces to be measured_m(x_m,y_m) P and P { P }_n(x_n,y_n)}；

S7, obtaining a coordinate set of the surface nerve contour point, the outer edge point of the hidden cavity window and the point of the curved surface to be measured in the three-dimensional space by the mapping relation of the step S5 through the two-dimensional image point of the surface nerve contour, the two-dimensional point of the outer edge of the hidden cavity window and the two-dimensional points of the two curved surfaces to be measured in the first visible light sensor

Step S8, filtering each three-dimensional point set once to remove outer points P_x；

Step S9, down-sampling the filtered point set, if the outer point P is_xWith its predecessor point P_x-1Or successor point P_x+1Is less than a certain distance d_tThen the outer point P is determined_xFiltering to obtain filtered three-dimensional point sets

Step S10, obtaining the length d of facial nerve by calculating the sum of the distances of adjacent points_m；

Step S11, passing point setObtaining a fitting plane, and collecting points Y_w' the points are projected onto the fitting plane, and then the filtering operation of step S8 and step S9 are performed to obtain a point setThereby obtaining an ideal fitting contour of the hidden window, obtaining the perimeter C of the hidden window by a method of solving the length of the line segment in a segmentation way, and further obtaining a point setCentral point p of_cAt the contour point setUpper connecting each point with a central point p_cDividing the recess window into a plurality of small triangles, and calculating the sum of the areas of the small triangles to obtain the area S of the recess window;

step S12, obtaining fitting planes pi 1 and pi 2 of each curved surface through the method in the step S11, calculating normal vectors l1(a1, b1 and c1) and l2(a2, b2 and c2) of the fitting planes, and obtaining a supplementary angle of an included angle between the two normal vectors, namely the included angle of the curved surfaces in the structures of the middle ear and the inner ear in the microsurgery.

7. The method for real-time three-dimensional measurement of key indicators in ear microsurgery according to claim 6, wherein the formula is specifically adopted in the step S5:wherein (x, y) represents an imageAt one point in the above-mentioned process,representing the parallax value at (X, Y) in the parallax map, and (X/W, Y/W, Z/W) representing the space coordinate of (X, Y) in the optical center coordinate system of the first visible light sensor.

8. The method of claim 6, wherein the outer point P of step S8 is the real-time three-dimensional measurement of the key index of ear microsurgery_xThe calculation formula is specifically as follows:

9. the method for real-time three-dimensional measurement of key indicators in ear microsurgery according to claim 6, wherein the calculation formula for calculating the sum of the distances of the neighboring points in step S10 is specifically as follows:

10. the method of claim 6, wherein the step S11 is performed by point collectionThe specific method for obtaining the fitting plane comprises the following steps:

step S111, constructing a fitting plane with an equation of Ax + Bx + Cz + D equal to 0;

step S112, according to the point setBy means of least squares:a fitted plane is obtained.

Technical Field

The invention relates to the technical field of computational microscopy imaging, in particular to a system and a method for real-time three-dimensional measurement of key indexes in an ear microsurgery.

Background

The microscope is the most commonly used auxiliary medical equipment in otolaryngology department, especially in ear microsurgery, benefits from the magnifying effect of microscope to the art field, and the doctor can carry out more refined operation treatment to ear disease patient.

At present, the information acquisition mode of a doctor on a microscopic operation field is mainly in a visual mode, and the processing of microscopic image information completely depends on the human brain. The microscopic image information includes cognitive identification of the operative field, intraoperative navigation planning, intraoperative measurements, and the like. Among these information processing mechanisms, the human brain is often relatively rough in the information processing measured intraoperatively, with large uncertainty. In the traditional technology, in order to solve the problem, otolaryngologists often use professional measuring scales to measure key indexes to be measured in an ear microsurgery operation field clinically, but the measuring scales can only measure a small amount of linear distances, and measurement of complex indexes such as the length of a three-dimensional irregular curve, the perimeter and the area of the three-dimensional irregular shape, the included angle of an irregular curved surface and the like is often unwise, and can only be roughly estimated by naked eyes.

Disclosure of Invention

The invention aims to provide a system and a method for real-time three-dimensional measurement of key indexes in an ear microsurgery, which can solve the problem that the key indexes in the current ear microsurgery are difficult to measure.

The embodiment of the invention is realized by the following steps:

in a first aspect, a system for real-time three-dimensional measurement of a key index in ear microsurgery includes:

the double-viewpoint microscopic imaging unit is used for generating microscopic image data of the symmetrical structure of the middle ear and the inner ear in the microscopic operation to be measured;

the double-viewpoint microscopic image acquisition unit is used for acquiring the microscopic image data;

the double-viewpoint microscopic image control and processing unit is used for controlling the shooting of the double-viewpoint microscopic image acquisition unit and processing the acquired microscopic image data to obtain a quantitative result of the index to be measured;

the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic imaging unit, and the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic image control and processing unit.

The invention relates to a system for real-time three-dimensional measurement of key indexes in an ear microsurgery, which generates microscopic images of two mutually symmetrical visual angles of a middle ear structure and an inner ear structure in the microsurgery to be measured through a double-viewpoint microscopic imaging unit, then controls a double-viewpoint microscopic image acquisition unit to acquire microscopic image data of the middle ear structure and the inner ear structure in the microsurgery to be measured through a double-viewpoint microscopic image control and processing unit, and finally processes the acquired microscopic image data to obtain a quantitative result of the indexes to be measured, so that the key indexes in the ear microsurgery can be obtained.

In some embodiments of the present invention, the dual-viewpoint microscopic imaging unit includes a light source, a large objective lens and a dual-optical path continuously variable volume, the large objective lens is connected to the light source, the light source is connected to the dual-optical path continuously variable volume, and the dual-optical path continuously variable volume is connected to the dual-viewpoint microscopic image collecting unit; the light source is used for illuminating the surgical field, the large objective lens is used for adjusting the imaging focal plane of the microscopic imaging device, and the double-optical-path continuously variable power body is used for changing the distance between the lenses in the optical lens group.

After the distance between the lenses in the optical lens group is changed by the double-light-path continuous zoom, the magnification of the operation field to be detected on the first visible light sensor and the second visible light sensor can be indirectly changed.

In some embodiments of the present invention, the dual-viewpoint microscopic image collecting unit includes a first visible light sensor and a second visible light sensor, the first visible light sensor provides a first viewing angle for the dual-viewpoint microscopic image collecting unit as a first observation optical path of middle ear and inner ear structures in the microscopic surgery to be measured, the second visible light sensor provides a second viewing angle for the dual-viewpoint microscopic image collecting unit as a second observation optical path of middle ear and inner ear structures in the microscopic surgery to be measured, and the first observation optical path and the second observation optical path are symmetrical.

In some embodiments of the present invention, the first visible light sensor and the second visible light sensor both receive light emitted from a surface of an operation field to be measured, and respectively present an image of the middle ear and inner ear structures at the first viewing angle and an image at the second viewing angle during a microscopic operation to be measured.

In some embodiments of the present invention, the dual-viewpoint microscopic image control and processing unit includes a synchronous trigger device, a dual-channel microscopic image acquisition card and a computer, the dual-viewpoint microscopic image acquisition unit is connected to the synchronous trigger device, the synchronous trigger device is connected to the dual-channel microscopic image acquisition card, and the dual-channel microscopic image acquisition card is connected to the computer; the synchronous trigger device is used for controlling the first visible light sensor and the second visible light sensor to synchronously shoot, the dual-channel microscopic image acquisition card is used for outputting data of the first visible light sensor and the second visible light sensor to the computer from a sensor end, and the computer is used for processing the dual-viewpoint microscopic image and calculating the quantification result of key indexes in structures of middle ear and inner ear in the microsurgery.

In a second aspect, the present application provides a method for real-time three-dimensional measurement of key indicators in ear microsurgery, which includes:

step S1, calibrating two visible light sensors in the double-viewpoint microscopic image acquisition unit, and acquiring internal parameters of the first visible light sensor and the second visible light sensor under each measurement magnificationAnd an external parameter T of the first visible light sensor relative to the second visible light sensor_i ¹²(ii) a Wherein subscript i is a particular microscopic imaging magnification;

step S2, adjusting the double-light-path continuous zoom body under a certain multiplying power i, and acquiring images of the middle ear and the inner ear structure parts in the operation field at the first visible light sensor by controlling the synchronous trigger deviceAnd images in the second visible light sensor

Step S3, based on internal and external parameters, using stereo correction algorithm in computer vision to correct image Correcting to obtain a corrected image of the first visible light sensorAnd a corrected image of the second visible light sensorAnd a corrected reprojection matrix Q of the first visible light sensor;

step S4, correcting the imagePerforming a dense matching algorithm to obtain a corrected imageOf (d) a parallax map₁₂；

Step S5, correcting imageAndreprojection matrix Q and disparity map d₁₂Obtaining corrected images using triangulation methods in computer visionThe space coordinates of each point in the model are under the optical center coordinate system of the first visible light sensor, and all the points form a point cloud model of the structures of the middle ear and the inner ear in the microsurgery in the operation field;

step S6, after the point cloud model is obtained, the image is correctedObtaining a two-dimensional image point set L { L } of the facial nerve outline by man-machine interactive circle selection or automatic identification model identification_i(x_i，y_i) Y and a two-dimensional set of points at the outer edge of the crypt window_j(x_j，y_j) And a set of two-dimensional points Q Q of the two curved surfaces to be measured_m(x_m，y_m) P and P { P }_n(x_n，y_n)}；

S7, obtaining a coordinate set of the surface nerve contour point, the outer edge point of the hidden cavity window and the point of the curved surface to be measured in the three-dimensional space by the mapping relation of the step S5 through the two-dimensional image point of the surface nerve contour, the two-dimensional point of the outer edge of the hidden cavity window and the two-dimensional points of the two curved surfaces to be measured in the first visible light sensor

Step S8, filtering each three-dimensional point set once to remove the outer point P_x；

Step S9, down-sampling the filtered point set, if the outer point P is_xWith its predecessor point P_x-1Or successor point P_x+1Is less than a certain distance d_tThen the outer point P is determined_xFiltering to obtain filtered three-dimensional point sets

Step S10, obtaining the length d of facial nerve by calculating the sum of the distances of adjacent points_m；

Step S11, passing point setObtaining a fitting plane, and collecting points into a set Y'_wThe points are projected on the fitting plane, and then the filtering operation of the step S8 and the step S9 are performed to obtain a point setThereby obtaining an ideal fitting contour of the hidden window, obtaining the perimeter C of the hidden window by a method of solving the length of the line segment in a segmentation way, and further obtaining a point setCentral point p of_cAt the contour point setUpper connecting each point with a central point p_cDividing the recess window into a plurality of small triangles, and calculating the sum of the areas of the small triangles to obtain the area S of the recess window;

step S12, obtaining fitting planes pi 1 and pi 2 of each curved surface through the method in the step S11, calculating normal vectors l1(a1, b1 and c1) and l2(a2, b2 and c2) of the fitting planes, and obtaining a supplementary angle of an included angle between the two normal vectors, namely the included angle of the curved surfaces in the structures of the middle ear and the inner ear in the microsurgery.

In some embodiments of the present invention, the above step S5 specifically uses the formula:wherein (x, y) represents an imageAt one point in the above-mentioned process,representing the parallax value at (X, Y) in the parallax map, and (X/W, Y/W, Z/W) representing the space coordinate of (X, Y) in the optical center coordinate system of the first visible light sensor。

In some embodiments of the present invention, the outer point P in the step S8 is_xThe calculation formula is specifically as follows:

in some embodiments of the present invention, the calculation formula for calculating the sum of the distances between the adjacent points in step S10 is specifically as follows:

in some embodiments of the present invention, the step S11 is performed by point collectionThe specific method for obtaining the fitting plane comprises the following steps:

step S111, constructing a fitting plane with an equation of Ax + Bx + Cz + D equal to 0;

step S112, according to the point setBy means of least squares:a fitted plane is obtained.

Compared with the prior art, the embodiment of the invention has at least the following advantages or beneficial effects:

the invention provides a system and a method for real-time three-dimensional measurement of key indexes in an ear microsurgery.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

FIG. 1 is a schematic diagram of a system for real-time three-dimensional measurement of key indicators in ear microsurgery in accordance with example 1 of the present invention;

FIG. 2 is a structural diagram of a dual viewpoint microscopic image acquisition unit of a system for real-time three-dimensional measurement of key indicators in ear microsurgery in accordance with embodiment 1 of the present invention;

FIG. 3 is a structural diagram of a dual viewpoint microscopic image control and processing unit of a system for real-time three-dimensional measurement of key indicators in ear microsurgery according to embodiment 1 of the present invention;

FIG. 4 is a block diagram of a system for real-time three-dimensional measurement of key indicators in ear microsurgery in accordance with example 1 of the present invention;

FIG. 5 is a step diagram of a method for real-time three-dimensional measurement of key indicators in ear microsurgery in accordance with example 2 of the present invention.

Reference numerals and descriptions: 111. double-light-path continuous metamploidy; 112. a light source; 113. a large objective lens; 121. a first visible light sensor; 122. a second visible light sensor; 131. a synchronous trigger device; 132. and (4) a computer.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

In the description of the present application, it is also to be noted that, unless otherwise explicitly specified or limited, the term "connected" is to be interpreted broadly, e.g. as a fixed connection, a detachable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.

Example 1

Referring to fig. 1 and 4, fig. 1 is a schematic diagram illustrating a system for real-time three-dimensional measurement of a key index in an ear microsurgery provided in embodiment 1 of the present application, and fig. 4 is a structural diagram illustrating the system for real-time three-dimensional measurement of a key index in an ear microsurgery provided in embodiment 1 of the present application.

The system for real-time three-dimensional measurement of key indexes in the ear microsurgery provided by the embodiment 1 of the application comprises:

the double-viewpoint microscopic imaging unit is used for generating microscopic image data of the symmetrical structure of the middle ear and the inner ear in the microscopic operation to be measured;

the double-viewpoint microscopic image acquisition unit is used for acquiring microscopic image data;

the double-viewpoint microscopic image control and processing unit is used for controlling the shooting of the double-viewpoint microscopic image acquisition unit and processing acquired microscopic image data to obtain a quantitative result of an index to be measured;

the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic imaging unit, and the double-viewpoint microscopic image acquisition unit is connected with the double-viewpoint microscopic image control and processing unit.

Specifically, in the system for real-time three-dimensional measurement of key indexes in ear microsurgery provided in embodiment 1 of the present application, when middle ear and inner ear structures in a measurement microsurgery are measured, first, microscopic image data symmetrical to the middle ear and inner ear structures in the measurement microsurgery are generated by the dual-viewpoint microimaging unit, then, the dual-viewpoint microimage control and processing unit controls the dual-viewpoint microimage acquisition unit to capture and acquire the microscopic image data, then, the dual-viewpoint microimage control and processing unit outputs the microscopic image data to the computer 132 for processing the microscopic image data, and a quantization result of the indexes to be measured is calculated by the computer 132.

As a preferred embodiment, the dual-viewpoint microscopic imaging unit comprises a light source 112, a large objective 113 and a dual-optical-path continuously variable ploidy 111, wherein the large objective 113 is connected with the light source 112, the light source 112 is connected with the dual-optical-path continuously variable ploidy 111, and the dual-optical-path continuously variable ploidy 111 is connected with the dual-viewpoint microscopic image acquisition unit; the light source 112 is used for illuminating the surgical field, the large objective 113 is used for adjusting the imaging focal plane of the microscopic imaging device, and the double-optical-path continuous zoom 111 is used for changing the distance between the lenses in the optical lens group.

Specifically, after the light source 112 provides illumination of the surgical field, the required imaging focal plane can be obtained by adjusting the large objective lens 113; the distance between the lenses in the optical lens group is changed through the double-light-path continuous zoom body 111, so that the magnification of the middle ear and inner ear structures on the first visible light sensor 121 and the second visible light sensor 122 in the microscopic operation to be detected can be changed.

It will be appreciated that the configuration shown in FIG. 1 is merely illustrative and that a system for real-time three-dimensional measurement of key indicators in ear microsurgery may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.

Referring to fig. 2, fig. 2 is a structural diagram of a dual-viewpoint microscopic image acquisition unit of a system for real-time three-dimensional measurement of key indicators in ear microsurgery according to embodiment 1 of the present invention.

As a preferred embodiment, the dual-viewpoint microscopic image collecting unit includes a first visible light sensor 121 and a second visible light sensor 122, the first visible light sensor 121 provides a first viewing angle for the dual-viewpoint microscopic image collecting unit as a first viewing optical path of middle ear and inner ear structures in the microscopic surgery to be measured, the second visible light sensor 122 provides a second viewing angle for the dual-viewpoint microscopic image collecting unit as a second viewing optical path of middle ear and inner ear structures in the microscopic surgery to be measured, and the first viewing optical path and the second viewing optical path are symmetrical.

Specifically, the first visible light sensor 121 provides a first viewing angle for the dual-viewpoint microscopic image acquisition unit, and is used for receiving light emitted from the surgical field surfaces of the middle ear and the inner ear structure in the detected microsurgery and finally presenting images of the middle ear and the inner ear structure in the detected microsurgery at the first viewing angle; the second visible light sensor 122 provides a second viewing angle for the dual-viewpoint microscopic image collecting unit, and is also used for receiving light emitted from the surgical field surface of the middle ear and inner ear structure in the detected microsurgery, and finally presenting an image of the middle ear and inner ear structure in the detected microsurgery at the second viewing angle.

Referring to fig. 3, fig. 3 is a structural diagram of a dual-viewpoint microscopic image control and processing unit of a system for real-time three-dimensional measurement of key indicators in ear microsurgery according to embodiment 1 of the present invention.

As a preferred embodiment, the dual-viewpoint microscopic image control and processing unit includes a synchronous trigger device 131, a dual-channel microscopic image acquisition card and a computer 132, the dual-viewpoint microscopic image acquisition unit is connected with the synchronous trigger device 131, the synchronous trigger device 131 is connected with the dual-channel microscopic image acquisition card, and the dual-channel microscopic image acquisition card is connected with the computer 132; the synchronous trigger device 131 is used for controlling the first visible light sensor 121 and the second visible light sensor 122 to synchronously photograph, the dual-channel microscopic image acquisition card is used for outputting data of the first visible light sensor 121 and the second visible light sensor 122 from the sensor ends to the computer 132, and the computer 132 is used for processing the dual-viewpoint microscopic image and calculating quantification results of key indexes in structures of the middle ear and the inner ear in the microsurgery.

Specifically, the synchronous trigger device 131 controls the first visible light sensor 121 and the second visible light sensor 122 in the dual-viewpoint microscopic image acquisition unit to synchronously shoot and acquire microscopic images, then the dual-channel microscopic image acquisition card outputs microscopic image data shot by the first visible light sensor 121 and the second visible light sensor 122 to the computer 132 from the sensor end, and finally the computer 132 processes the dual-viewpoint microscopic images, so that the quantification results of key indexes in the structures of the middle ear and the inner ear in the microsurgery are calculated.

Example 2

Referring to fig. 5, fig. 5 is a step diagram of a method for real-time three-dimensional measurement of key indicators in ear microsurgery according to embodiment 2 of the present invention.

The method for real-time three-dimensional measurement of key indexes in the ear microsurgery provided by the embodiment 2 of the application is characterized by comprising the following steps:

step S1, calibrating two visible light sensors in the double-viewpoint microscopic image acquisition unit, and acquiring internal parameters of the first visible light sensor and the second visible light sensor under each measurement magnificationAnd an external parameter T of the first visible light sensor relative to the second visible light sensor_i ¹²(ii) a Wherein subscript i is a particular microscopic imaging magnification;

step S2, adjusting the double-light-path continuous zoom body under a certain multiplying power i, and acquiring images of the middle ear and the inner ear structure parts in the operation field at the first visible light sensor by controlling the synchronous trigger deviceAnd images in the second visible light sensor

Step S3, based on internal and external parameters, using stereo correction algorithm in computer vision to correct image Correcting to obtain a corrected image of the first visible light sensorAnd a corrected image of the second visible light sensorAnd a corrected reprojection matrix Q of the first visible light sensor;

step S4, correcting the imagePerforming a dense matching algorithm to obtain a corrected imageOf (d) a parallax map₁₂；

Step S5, correcting imageAndreprojection matrix Q and disparity map d₁₂Obtaining corrected images using triangulation methods in computer visionThe space coordinates of each point in the model are under the optical center coordinate system of the first visible light sensor, and all the points form a point cloud model of the structures of the middle ear and the inner ear in the microsurgery in the operation field;

specifically, the formula is adopted:wherein (x, y) represents an imageAt one point in the above-mentioned process,representing the parallax value at (X, Y) in the parallax map, and (X/W, Y/W, Z/W) representing the space coordinate of (X, Y) in the optical center coordinate system of the first visible light sensor.

Step S6, after the point cloud model is obtained, the image is correctedObtaining a two-dimensional image point set L { L } of the facial nerve outline by man-machine interactive circle selection or automatic identification model identification_i(x_i，y_i) Y and a two-dimensional set of points at the outer edge of the crypt window_j(x_j，y_j) And a set of two-dimensional points Q Q of the two curved surfaces to be measured_m(x_m，y_m) P and P { P }_n(x_n，y_n)}；

S7, obtaining a coordinate set of the surface nerve contour point, the outer edge point of the hidden cavity window and the point of the curved surface to be measured in the three-dimensional space by the mapping relation of the step S5 through the two-dimensional image point of the surface nerve contour, the two-dimensional point of the outer edge of the hidden cavity window and the two-dimensional points of the two curved surfaces to be measured in the first visible light sensor

Step S8, filtering each three-dimensional point set once to remove the outer point P_x；

Wherein if point P_xWith its predecessor point P_x-1After, thenPoint P_x+1Is greater than a times the average distance of the successive points preceding and succeeding the respective points, P is considered to be_xIs the outer point.

In particular, the outer point P_xThe calculation formula of (2) is as follows:

step S9, down-sampling the filtered point set, if the outer point P is_xWith its predecessor point P_x-1Or successor point P_x+1Is less than a certain distance d_tThen the outer point P is determined_xFiltering to obtain filtered three-dimensional point sets

Step S10, obtaining the length d of facial nerve by calculating the sum of the distances of adjacent points_m；

Specifically, the calculation formula for calculating the sum of distances between adjacent points is specifically:

step S11, passing point setObtaining a fitting plane, and collecting points into a set Y'_wThe points are projected on the fitting plane, and then the filtering operation of the step S8 and the step S9 are performed to obtain a point setThereby obtaining an ideal fitting contour of the hidden window, obtaining the perimeter C of the hidden window by a method of solving the length of the line segment in a segmentation way, and further obtaining a point setCentral point p of_cIn aSet of contour pointsUpper connecting each point with a central point p_cDividing the recess window into a plurality of small triangles, and calculating the sum of the areas of the small triangles to obtain the area S of the recess window;

in particular, by point collectionThe method for acquiring the fitting plane comprises the following steps:

step S111, constructing a fitting plane with an equation of Ax + Bx + Cz + D equal to 0;

step S112, according to the point setBy means of least squares:a fitted plane is obtained.

Step S12, obtaining fitting planes pi 1 and pi 2 of each curved surface through the method in the step S11, calculating normal vectors l1(a1, b1 and c1) and l2(a2, b2 and c2) of the fitting planes, and obtaining a supplementary angle of an included angle between the two normal vectors, namely the included angle of the curved surfaces in the structures of the middle ear and the inner ear in the microsurgery.

Embodiment 2 of the present application provides a method for real-time three-dimensional measurement of a key index in an ear microsurgery based on the system for real-time three-dimensional measurement of a key index in an ear microsurgery in embodiment 1, by calibrating two visible light sensors in a dual-viewpoint microscopic image acquisition unit, thereby obtaining internal and external parameters thereof, then by controlling a synchronous trigger device, images of structural parts of the middle ear and the inner ear in a measured surgical field on the two visible light sensors can be obtained, and then by correcting the images by using a stereo correction algorithm in computer vision, points with the same characteristics in the two images can be aligned, that is, a pair of corrected images and a reprojection matrix of a first visible light sensor can be obtained, and by executing the algorithm, parallax error of the two pairs of images can be obtainedObtaining the space coordinates of each point in the corrected image under the optical center coordinate system corresponding to the visible light sensor by using a triangulation method in computer vision, wherein all space points can form a point cloud model, the corrected image can be identified by a man-machine interactive circle selection or an automatic identification model to obtain a two-dimensional image point set of a surface nerve outline, a two-dimensional point set of the outer edge of a hidden window and two-dimensional point sets of two curved surfaces to be measured, obtaining a coordinate set of the surface nerve outline point, the outer edge point of the hidden window and the curved surface point to be measured in a three-dimensional space by a mapping relation, filtering, removing the outer points, performing down-sampling to obtain three-dimensional point sets, calculating the sum of the distances of adjacent points to obtain the length of the surface nerve, and obtaining the length of the surface nerve by point set combinationThe fitting plane can be obtained, then the ideal fitting contour of the recess window can be obtained through wave filtering operation, the perimeter C of the recess window can be obtained through a method of solving the length of the line segment in a segmented mode, the area S of the recess window can be obtained through calculation, normal vectors of the fitting plane can be obtained through the fitting plane of each curved surface, and the supplementary angle of the included angle of the two normal vectors can be obtained, so that the included angle of the curved surfaces in the structures of the middle ear and the inner ear in the measured microsurgery can be obtained.

Therefore, the method can measure key complex indexes such as the length of facial nerves, the perimeter and the area of a crypt window in the field of the ear microsurgery, curved surface included angles in structures of a middle ear and an inner ear in the measured microsurgery and the like.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus or method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In summary, the system and method for real-time three-dimensional measurement of key indexes in ear microsurgery provided by the embodiment of the application generate microscopic images of two mutually symmetrical viewing angles of structures of middle ear and inner ear in the microsurgery to be measured through the dual-viewpoint microimaging unit, then control the dual-viewpoint microimage acquisition unit to acquire the microimage data of the structures of middle ear and inner ear in the microsurgery to be measured through the dual-viewpoint microimage control and processing unit, and finally process the acquired microimage data to obtain a quantization result of the indexes to be measured, so that the key indexes in the ear microsurgery can be obtained, and the effect of measuring complex indexes can be obtained.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.