阅读说明：本技术 角膜共聚焦图像中菌丝识别方法及装置 (Method and device for recognizing hyphae in corneal confocal image ) 是由 洪晶 秦晓冉 彭荣梅 程健 陈炜 于 2021-07-08 设计创作，主要内容包括：本发明提供一种角膜共聚焦图像中菌丝识别方法及装置,该方法包括：获取待识别的角膜共聚焦图像；将所述角膜共聚焦图像输入到菌丝-神经纤维分割模型,根据所述菌丝-神经纤维分割模型输出预分割结果图；对预分割结果图进行自动修正得到最终分割结果图；根据所述最终分割结果图判断是否存在菌丝。本发明提供的角膜共聚焦图像中菌丝识别方法及装置,通过将角膜共聚焦图像输入到菌丝-神经纤维分割模型,根据菌丝-神经纤维分割模型输出预分割结果图,对预分割结果图进行自动修正得到最终分割结果图,根据最终分割结果图判断是否存在菌丝,实现对图像中菌丝和神经纤维的准确区分,由此可以实现角膜共聚焦图像中菌丝的自动化准确识别。(The invention provides a method and a device for recognizing hyphae in a corneal confocal image, wherein the method comprises the following steps: acquiring a corneal confocal image to be identified; inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and judging whether hyphae exist according to the final segmentation result graph. According to the method and the device for recognizing the hyphae in the corneal confocal image, the corneal confocal image is input into the hypha-nerve fiber segmentation model, the pre-segmentation result image is output according to the hypha-nerve fiber segmentation model, the pre-segmentation result image is automatically corrected to obtain the final segmentation result image, and whether the hyphae exist is judged according to the final segmentation result image, so that the hyphae and the nerve fiber in the image are accurately distinguished, and the automatic and accurate recognition of the hyphae in the corneal confocal image can be realized.)

1. A method for recognizing hyphae in a corneal confocal image is characterized by comprising the following steps:

acquiring a corneal confocal image to be identified;

inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training;

automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph;

and judging whether hyphae exist according to the final segmentation result graph.

2. The method for identifying hyphae in a confocal image of cornea according to claim 1, wherein before the acquiring the confocal image of cornea to be identified, the method further comprises:

obtaining the corneal confocal image sample, and respectively depicting along the central line of hyphae and/or nerve fibers based on the corneal confocal image sample to obtain a region labeling image sample; wherein the pixels representing hyphae and nerve fibers in the region labeling image sample have different pixel values.

3. The method for recognizing hyphae in a confocal image of a cornea according to claim 1, wherein the automatically correcting the pre-segmentation result map to obtain a final segmentation result map specifically comprises:

obtaining each hypha segment and each nerve fiber segment in the pre-segmentation result graph;

and calculating preset characteristic indexes of the hypha segments and the nerve fiber segments, comparing the preset characteristic indexes with thresholds of the preset characteristic indexes, and performing class correction on the hypha segments and the nerve fiber segments in the pre-segmentation result graph according to a comparison result to obtain a final segmentation result graph.

4. The method for identifying hyphae in a confocal image of a cornea according to claim 3, wherein the preset characteristic index comprises at least one of an angle between branches at a cross point in a segment, a segment length and a segment curvature.

5. The method for hyphal identification in a confocal image of a cornea according to claim 3, further comprising:

and superposing the final segmentation result image and the corneal confocal image to obtain a visual image, and displaying the visual image.

6. The method for recognizing hyphae in a confocal image of cornea according to claim 1, wherein the inputting the confocal image of cornea into a hyphae-nerve fiber segmentation model and outputting a pre-segmentation result map according to the hyphae-nerve fiber segmentation model specifically comprises:

inputting the corneal confocal image into a skeleton network block of an encoder, and outputting a first characteristic diagram; inputting the first feature map into a large core separable volume block of the encoder, and outputting a second feature map; inputting the second feature map into a self-attention block of the encoder, and outputting a third feature map; and inputting the third feature map into a decoder, and outputting the pre-segmentation result map.

7. The method for recognizing hyphae in a confocal image of cornea according to claim 6, wherein a part of deep-layer convolution layers of the skeleton network block is adjusted to be deformable convolution layers;

and the large kernel separable convolution block divides the input first feature map into two branches, one branch is convolved by a series of 1 × k convolution products and k × 1, the other branch is convolved by a series of k × 1 convolution products and 1 × k, and then the feature maps obtained by the two branches are added point by point to obtain the second feature map.

8. A device for recognizing hyphae in a confocal image of a cornea, comprising:

an image acquisition module to: acquiring a corneal confocal image to be identified;

a hypha pre-segmentation module to: inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training;

the pre-segmentation result automatic correction module is used for: automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph;

a hyphal recognition module to: and judging whether hyphae exist according to the final segmentation result graph.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for hyphal identification in a confocal image of cornea according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the method for hyphal identification in a confocal image of cornea as claimed in any one of claims 1 to 7.

Technical Field

The invention relates to the technical field of medical image processing, in particular to a method and a device for recognizing hyphae in a corneal confocal image.

Background

The appearance of hyphae in a corneal confocal image is an important basis for judging fungal keratitis, but sometimes hyphae and nerve fibers in the corneal confocal image exist simultaneously, are linear structures, are similar in shape and difficult to distinguish, and need an experienced ophthalmologist to find the difference between the hyphae and the nerve fibers and distinguish the hyphae and the nerve fibers carefully, so that missed diagnosis and misdiagnosis can be caused due to different subjective experiences of different doctors. The prior art does not consider a distinguishing technology of hyphae and nerve fibers, but the distinguishing technology is the difficulty of hyphae identification.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a method and a device for identifying hyphae in a corneal confocal image.

The invention provides a method for recognizing hyphae in a corneal confocal image, which comprises the following steps: acquiring a corneal confocal image to be identified; inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and judging whether hyphae exist according to the final segmentation result graph.

According to the method for identifying the hyphae in the corneal confocal image provided by the invention, before the acquisition of the corneal confocal image to be identified, the method further comprises the following steps: obtaining the corneal confocal image sample, and respectively depicting along the central line of hyphae and/or nerve fibers based on the corneal confocal image sample to obtain a region labeling image sample; wherein the pixels representing hyphae and nerve fibers in the region labeling image sample have different pixel values.

According to the method for recognizing hyphae in the corneal confocal image, provided by the invention, the automatic correction is performed on the pre-segmentation result image to obtain a final segmentation result image, and the method specifically comprises the following steps: obtaining each hypha segment and each nerve fiber segment in the pre-segmentation result graph; and calculating preset characteristic indexes of the hypha segments and the nerve fiber segments, comparing the preset characteristic indexes with thresholds of the preset characteristic indexes, and performing class correction on the hypha segments and the nerve fiber segments in the pre-segmentation result graph according to a comparison result to obtain a final segmentation result graph.

According to the method for recognizing the hyphae in the corneal confocal image, the preset characteristic indexes comprise at least one of angles among branches at cross points in the segments, lengths of the segments and curvatures of the segments.

According to the invention, the method for recognizing the hyphae in the corneal confocal image further comprises the following steps: and superposing the final segmentation result image and the corneal confocal image to obtain a visual image, and displaying the visual image.

According to the method for recognizing the hyphae in the corneal confocal image, the corneal confocal image is input into a hypha-nerve fiber segmentation model, and a pre-segmentation result image is output according to the hypha-nerve fiber segmentation model, and the method specifically comprises the following steps: inputting the corneal confocal image into a skeleton network block of an encoder, and outputting a first characteristic diagram; inputting the first feature map into a large core separable volume block of the encoder, and outputting a second feature map; inputting the second feature map into a self-attention block of the encoder, and outputting a third feature map; and inputting the third feature map into a decoder, and outputting the pre-segmentation result map.

According to the method for recognizing the hyphae in the corneal confocal image, provided by the invention, part of deep-layer convolution layers of the skeleton network block are adjusted into deformable convolution layers; and the large kernel separable convolution block divides the input first feature map into two branches, one branch is convolved by a series of 1 × k convolution products and k × 1, the other branch is convolved by a series of k × 1 convolution products and 1 × k, and then the feature maps obtained by the two branches are added point by point to obtain the second feature map.

The invention also provides a hypha recognition device in a corneal confocal image, comprising: an image acquisition module to: acquiring a corneal confocal image to be identified; a hypha pre-segmentation module to: inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; the pre-segmentation result automatic correction module is used for: automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; a hyphal recognition module to: and judging whether hyphae exist according to the final segmentation result graph.

The invention further provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the above methods for identifying hyphae in the corneal confocal image.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, implements the steps of the method for hypha identification in a confocal image of cornea as described in any one of the above.

According to the method and the device for recognizing the hyphae in the corneal confocal image, the corneal confocal image is input into the hypha-nerve fiber segmentation model, the pre-segmentation result image is output according to the hypha-nerve fiber segmentation model, the pre-segmentation result image is automatically corrected to obtain the final segmentation result image, and whether the hyphae exist is judged according to the final segmentation result image, so that the hyphae and the nerve fiber in the image are accurately distinguished, and the automatic and accurate recognition of the hyphae in the corneal confocal image can be realized.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a flow chart of a method for identifying hyphae in a confocal image of a cornea according to the present invention;

FIG. 2 is a second flowchart of a method for identifying hyphae in a confocal image of a cornea according to the present invention;

FIG. 3 is a schematic structural diagram of a hypha recognition device in a confocal image of a cornea according to the present invention;

fig. 4 illustrates a physical structure diagram of an electronic device.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The method and device for identifying hyphae in a corneal confocal image according to the present invention will be described with reference to fig. 1 to 4.

Fig. 1 is a flowchart of a method for identifying hyphae in a confocal image of a cornea according to the present invention. As shown in fig. 1, the method includes:

step 101, acquiring a corneal confocal image to be identified;

102, inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training;

103, automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph;

and 104, judging whether hyphae exist according to the final segmentation result graph.

First, a confocal image of the cornea to be identified, which is taken by means of a confocal microscope, is acquired. And then inputting the corneal confocal image into a hypha-nerve fiber segmentation model trained in advance, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model. The hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training. The output labels corresponding to all pixel points of the corneal confocal image sample are set according to the region labeling image sample, wherein the region labeling image sample is obtained by performing hypha region labeling and/or nerve fiber region labeling on the corneal confocal image sample. For example, the output label of the background-class pixel may be set to 0, the output label of the hypha-class pixel may be set to 1, and the output label of the nerve fiber-class pixel may be set to 2.

When a hypha-nerve fiber segmentation model is trained, a corneal confocal image sample and a region labeling image sample are firstly obtained for machine learning training. After all the corneal confocal images to be used for machine learning training are collected, a professional doctor firstly screens the images according to the image quality to select high-definition images containing hyphae and nerve fibers (both or at least one of the hyphae and the nerve fibers). And then, respectively labeling a hypha area and a nerve fiber area in the image by a plurality of professional doctors trained by a hypha and nerve fiber distinguishing method, thereby obtaining a region labeled image sample. Of course, if no hyphal region exists, no hyphal region is labeled, and if no nerve fiber region exists, no nerve fiber region is labeled. If a confocal image of the cornea includes both a hyphal region and a nerve fiber region, both the hyphal region and the nerve fiber region are labeled. The hypha region and the nerve fiber region in the region labeling image sample are to be distinguished, for example, the hypha region and the nerve fiber region are respectively represented by different pixel values. And then, taking the corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha area, a nerve fiber area or a background area as an output label, and performing machine learning training to obtain a hypha-nerve fiber segmentation model.

When hypha recognition is carried out in the corneal confocal image, the corneal confocal image is input into a hypha-nerve fiber segmentation model, and a pre-segmentation result graph which simultaneously provides a hypha region and a nerve fiber region is output. And each pixel point in the pre-segmentation result image is predicted to be a background class, a hypha class or a nerve fiber class. Wherein, the pixel points of the hypha area are hypha types, and the pixel points of the nerve fiber area are nerve fibers. And pixel points of other areas of the non-hyphal area and the non-nerve fiber area are in the background class. And automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph, for example, automatically correcting the category of pixel points in the pre-segmentation result graph to obtain the final segmentation result graph. And (4) recognizing hyphae in the corneal confocal image according to whether the final segmentation result graph contains hyphae regions.

The method for identifying the hyphae in the corneal confocal image comprises the steps of inputting the corneal confocal image into a hypha-nerve fiber segmentation model, outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model, automatically correcting the pre-segmentation result image to obtain a final segmentation result image, and judging whether the hyphae exist according to the final segmentation result image to accurately distinguish the hyphae from nerve fibers in the image, so that the hyphae in the corneal confocal image can be automatically and accurately identified.

According to the method for identifying the hyphae in the corneal confocal image provided by the invention, before the acquisition of the corneal confocal image to be identified, the method further comprises the following steps: obtaining the corneal confocal image sample, and respectively depicting along the central line of hyphae and/or nerve fibers based on the corneal confocal image sample to obtain a region labeling image sample; wherein the pixels representing hyphae and nerve fibers in the region labeling image sample have different pixel values.

Before hypha recognition is performed by using the hypha-nerve fiber segmentation model, the hypha-nerve fiber segmentation model needs to be obtained through machine learning training. Before training a hypha-nerve fiber segmentation model, a corneal confocal image sample and a region labeling image sample are acquired. The cornea confocal image sample can be directly obtained by taking a cornea image through a confocal microscope. The area labeling image sample is obtained by performing hypha area labeling and/or nerve fiber area labeling on the corneal confocal image sample. When hypha region labeling and/or nerve fiber region labeling are/is performed on the corneal confocal image sample, the hypha region labeling is performed by drawing along the center line of hypha, and the nerve fiber region labeling is performed by drawing along the center line of nerve fiber, so that the region-labeled image sample is obtained. The annotation tool may use the GIMP image processing software.

For example, pixels representing the hyphae and the nerve fibers in the region labeling image sample have different pixel values to distinguish the hyphae region from the nerve fiber region.

After the labeling is finished, a corneal confocal image sample library containing hypha region labeling and nerve fiber region labeling is constructed, and image data are randomly divided into a training data set and a verification data set according to a preset proportion and used for training and verifying a model. The sample library can be used for hypha recognition and segmentation of a corneal confocal image, nerve fiber recognition and segmentation and related basic index calculation work. In this embodiment, the method is used for training a hypha-nerve fiber segmentation model.

According to the method for identifying the hypha in the corneal confocal image, the corneal confocal image sample is obtained, and the area labeling image sample is obtained by respectively describing the corneal confocal image sample along the central line of the hypha and/or the nerve fiber, so that the accuracy of labeling the hypha area and the nerve fiber area is improved.

According to the method for recognizing hyphae in the corneal confocal image, provided by the invention, the automatic correction is performed on the pre-segmentation result image to obtain a final segmentation result image, and the method specifically comprises the following steps: obtaining each hypha segment and each nerve fiber segment in the pre-segmentation result graph; and calculating preset characteristic indexes of the hypha segments and the nerve fiber segments, comparing the preset characteristic indexes with thresholds of the preset characteristic indexes, and performing class correction on the hypha segments and the nerve fiber segments in the pre-segmentation result graph according to a comparison result to obtain a final segmentation result graph.

After the hypha and nerve fiber region extraction is performed by using the hypha-nerve fiber segmentation model, the result correction can be further performed on the pre-segmentation result graph. The result correction can be performed by using the characteristic indexes of hypha segmentation and nerve fiber segmentation. The specific method can be that hypha segments and nerve fiber segments in the pre-segmentation result graph are obtained, preset characteristic indexes of the hypha segments and the nerve fiber segments are calculated, threshold values of the preset characteristic indexes and threshold values of the preset characteristic indexes are compared, and classification correction is carried out on the hypha segments and the nerve fiber segments in the pre-segmentation result graph according to the comparison result to obtain a final segmentation result graph.

The threshold of the preset characteristic index can be obtained empirically or according to the statistical value of the sample in the training process. When the threshold value of the preset characteristic index is set, the threshold value of the preset characteristic index corresponding to the hypha is obtained by carrying out statistical analysis on the preset characteristic index of the hypha segment; and carrying out statistical analysis on the preset characteristic indexes of the nerve fiber segments to obtain a threshold value corresponding to the preset characteristic indexes of the nerve fibers. Therefore, the threshold value of the preset characteristic index includes a threshold value of the preset characteristic index of the hyphae and a threshold value of the preset characteristic index of the nerve fiber. The threshold value of the preset characteristic index of the hyphae and the threshold value of the preset characteristic index of the nerve fibers can be a single value or a range.

When the pre-segmentation result graph output by the hypha-nerve fiber segmentation model is modified, the preset characteristic indexes of the hypha segmentation and the nerve fiber segmentation are calculated, the obtained preset characteristic indexes can be respectively compared with the threshold value of the preset characteristic index of the hypha and the threshold value of the preset characteristic index of the nerve fiber, and the actual preset characteristic index belongs to which threshold value range, the corresponding category is modified. For example, after a certain hypha is calculated in a segmented manner to obtain a preset characteristic index, the obtained preset characteristic index is compared with a threshold of the preset characteristic index of the hypha, and the actually calculated preset characteristic index is found not to meet the threshold range of the preset characteristic index of the hypha; and after comparing the obtained preset characteristic index with the threshold value of the preset characteristic index of the nerve fiber, if the actually calculated preset characteristic index meets the threshold value range of the preset characteristic index of the nerve fiber, modifying the category of the corresponding hypha segment into the nerve fiber category.

According to the method for recognizing the hypha in the corneal confocal image, the hypha segmentation and the nerve fiber segmentation in the pre-segmentation result image are obtained, the preset characteristic indexes of the hypha segmentation and the nerve fiber segmentation are calculated, the preset characteristic indexes are compared with the threshold value of the preset characteristic indexes, and the hypha segmentation and the nerve fiber segmentation in the pre-segmentation result image are subjected to class correction according to the comparison result to obtain the final segmentation result image, so that the accuracy of the segmentation result is further improved.

According to the method for recognizing the hyphae in the corneal confocal image, the preset characteristic indexes comprise at least one of angles among branches at cross points in the segments, lengths of the segments and curvatures of the segments.

On the basis of a pre-segmentation result graph output by a hypha-nerve fiber segmentation model, calculating characteristic indexes of hypha segments and nerve fiber segments in the pre-segmentation result, including at least one of angles between branches at cross points in the segments, segment lengths and segment curvatures, setting a threshold value for each characteristic index according to statistical information in labeled data, and then respectively judging whether each characteristic index result meets the threshold value requirement, so as to perform class correction on the hypha segments and the nerve fiber segments in the pre-segmentation result to obtain a final segmentation result graph of the hypha and the nerve fiber.

Under the condition of setting a plurality of preset characteristic indexes, the result correction can be carried out under the condition that all the preset characteristic indexes meet corresponding threshold values.

According to the method for identifying the hyphae in the corneal confocal image, the accuracy of the segmentation result is further improved by setting the preset characteristic indexes to include at least one of the angle between branches, the length of the segments and the curvature of the segments at the cross points in the segments.

According to the invention, the method for recognizing the hyphae in the corneal confocal image further comprises the following steps: and superposing the final segmentation result image and the corneal confocal image to obtain a visual image, and displaying the visual image.

In the final segmentation result map, the hyphal region and the nerve fiber region can be displayed in a differentiated manner by different pixel values, and the pixel value of the background-class pixel can be set to 0. The final segmentation result graph and the input corneal confocal image to be identified can be superposed to obtain a visual image, and the visual image is displayed. In the visual image, the hypha area and the nerve fiber area can be displayed at the same time, and only the hypha area or only the nerve fiber area can be displayed according to the requirement.

Through the processing, a final segmentation result graph for distinguishing hyphae and nerve fibers in the corneal confocal image to be identified can be obtained, and an identification result can be given according to whether a hyphae region exists in the final segmentation result graph; in addition, the final segmentation result image and the image to be diagnosed are superposed to obtain a visual image. Hypha recognition results and visual images can be given on the detection report, so that visual display can be realized through hypha display areas.

According to the method for recognizing the hyphae in the corneal confocal image, the visual image is obtained by superposing the final segmentation result image and the corneal confocal image, and is displayed, so that the intuitiveness of the recognition result display is improved.

According to the method for recognizing the hyphae in the corneal confocal image, the corneal confocal image is input into a hypha-nerve fiber segmentation model, and a pre-segmentation result image is output according to the hypha-nerve fiber segmentation model, and the method specifically comprises the following steps: inputting the corneal confocal image into a skeleton network block of an encoder, and outputting a first characteristic diagram; inputting the first feature map into a large core separable volume block of the encoder, and outputting a second feature map; inputting the second feature map into a self-attention block of the encoder, and outputting a third feature map; and inputting the third feature map into a decoder, and outputting the pre-segmentation result map.

The hyphal-nerve fiber segmentation model includes an encoder and a decoder. The encoder extracts image features by using a skeleton network block, a large kernel separable convolution block and a self-attention block, and outputs a down-sampling feature map. And part of deep convolutional layers of the skeleton network block are adjusted into deformable convolutional layers, the deformable convolutional layers learn an offset on each sampling point of the input characteristic diagram by using a parallel network, the sizes of the adaptive learning receptive fields (the traditional convolution is the fixed receptive field size) are concentrated in an interested region or a target, and the adaptive convolutional layers are more suitable for objects with variable scales and variable geometric deformation such as hyphae and nerve fibers. The large-kernel separable convolution block divides an input feature graph into two branches, one branch is convolved by a series-connected 1 x k convolution and k x 1, the other branch is convolved by a series-connected k x 1 convolution and 1 x k convolution, then the feature graphs obtained by the two branches are added point by point to obtain an output feature graph, and the large-kernel separable convolution can obtain a large receptive field under the condition of small calculation amount. k is typically a positive integer greater than 3. The self-attention block adaptively learns attention weights, weights the input feature map, and makes the encoder focus more on global features related to hyphae and nerve fibers. The decoder comprises an deconvolution layer, a convolution layer and a softmax layer, wherein the deconvolution layer performs feature extraction on a feature map output by the decoder in a layer-by-layer upsampling mode and outputs an upsampled feature map. Finally, the decoder performs dimension transformation on the feature maps output by the deconvolution layer through a convolution layer, and outputs the feature maps with the dimension number equal to the category number according to the category number. The Softmax layer is used for normalizing the feature vector of each pixel position on the input feature map to obtain the probability that each pixel is a background class, a hyphal class or a nerve fiber class, and the class with the maximum probability value is used as the class of the corresponding pixel. The decoder can adopt a U-Net jump connection structure, in the up-sampling process, the feature graph output by the deconvolution layer is spliced with the feature graph with the corresponding size in the encoding process, and the spliced feature graph can realize the fusion of shallow features and deep features through a plurality of convolution layers.

The framework network block may use a framework network of various existing convolutional neural networks, including but not limited to using VGG, ResNet, densnet, etc.

The method for recognizing the hyphae in the corneal confocal image provided by the invention has the advantages that the functions of a hypha-nerve fiber segmentation model are realized by extracting the characteristics by utilizing the encoder comprising the skeleton network block, the large kernel separable convolution block and the self-attention block and outputting a pre-segmentation result image by utilizing the decoder.

The method for recognizing the hyphae in the corneal confocal image is characterized in that part of deep-layer convolution layers of the skeleton network block are adjusted into deformable convolution layers; and the large kernel separable convolution block divides the input first feature map into two branches, one branch is convolved by a series of 1 × k convolution products and k × 1, the other branch is convolved by a series of k × 1 convolution products and 1 × k, and then the feature maps obtained by the two branches are added point by point to obtain the second feature map.

Aiming at considering the differences of the hyphae and the nerve fibers in continuity, length, branch angles and the like, the hypha-nerve fiber segmentation model structure is mainly characterized in that the hyphae and the nerve fibers have various scales and variable geometric deformation at the first point; second, elongated morphology of hyphae and nerve fibers; and thirdly, paying attention to the characteristics of a larger peripheral range and increasing the receptive field of the model. And adjusting part of deep convolutional layers of the skeleton network block into deformable convolutional layers, wherein the deformable convolutional layers learn an offset on each sampling point of the input characteristic diagram through a parallel network, and the adaptive learning of the size of the receptive field is concentrated in an interested region or a target. Using large kernel separable volume blocks, a large receptive field is obtained without the computationally intensive k x 1 and 1 x k convolutions in the large kernel separable volume blocks.

According to the method for recognizing the hyphae in the corneal confocal image, provided by the invention, the skeleton network block is adjusted into the deformable convolution layer, and the large-kernel separable convolution block is constructed by utilizing k × 1 convolution and 1 × k convolution, so that the function realization of the hyphae-nerve fiber is further ensured.

The invention provides a method for identifying hyphae in a corneal confocal image, which can accurately distinguish the hyphae and the nerve fibers in the image by extracting two areas of the hyphae and the nerve fibers and automatically correcting an extraction result, thereby accurately identifying the hyphae in the image. In addition, a hypha detection result report can be generated, hypha and nerve fiber areas in the image can be displayed visually, and the visual examination result can be provided for doctors.

FIG. 2 is a second flowchart of the method for identifying hyphae in a confocal image of cornea according to the present invention. The method for recognizing hyphae in the corneal confocal image comprises the following steps:

s1, acquiring a corneal confocal image to be identified;

s2, inputting the corneal confocal image to be recognized into a trained hypha-nerve fiber segmentation model to obtain a pre-segmentation result graph of hypha and nerve fiber;

s3, calculating characteristic indexes of hypha segments and nerve fiber segments based on the pre-segmentation result graph, comparing each characteristic index with a threshold value obtained by statistical information, and automatically correcting the extraction result to obtain a final segmentation result graph of hypha and nerve fiber;

and S4, identifying whether a hypha region exists in the corneal confocal image to be identified based on the final segmentation result image, and giving an identification result of hypha in the corneal confocal image.

The method for identifying the hyphae in the corneal confocal image distinguishes the hyphae and the nerve fibers to obtain an accurate hyphae area, so that the hyphae in the corneal confocal image can be accurately identified.

The device for identifying hyphae in a confocal image of cornea provided by the present invention is described below, and the device for identifying hyphae in a confocal image of cornea described below and the method for identifying hyphae in a confocal image of cornea described above can be referred to in correspondence.

Fig. 3 is a schematic structural diagram of a hypha recognition device in a confocal image of cornea provided by the invention. As shown in fig. 3, the apparatus includes an image acquisition module 10, a hypha pre-segmentation module 20, a pre-segmentation result automatic correction module 30, and a hypha identification module 40, wherein: the image acquisition module 10 is configured to: acquiring a corneal confocal image to be identified; the hypha pre-segmentation module 20 is used for: inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; the pre-segmentation result automatic correction module 30 is used for automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and the hypha recognition module 40 is used for judging whether hypha exists according to the final segmentation result graph.

The device for identifying the hyphae in the corneal confocal image provided by the invention inputs the corneal confocal image into the hypha-nerve fiber segmentation model, outputs the pre-segmentation result image according to the hypha-nerve fiber segmentation model, automatically corrects the pre-segmentation result image to obtain the final segmentation result image, and judges whether the hyphae exist according to the final segmentation result image, so that the hyphae and nerve fibers in the image can be accurately distinguished, and the automatic accurate identification of the hyphae in the corneal confocal image can be realized.

According to the hypha recognition device in the corneal confocal image provided by the invention, the device further comprises a sample construction module, wherein the sample construction module is used for: obtaining the corneal confocal image sample, and respectively depicting along the central line of hyphae and/or nerve fibers based on the corneal confocal image sample to obtain a region labeling image sample; wherein the pixels representing hyphae and nerve fibers in the region labeling image sample have different pixel values.

According to the device for identifying the hypha in the corneal confocal image, the corneal confocal image sample is obtained, and the area labeling image sample is obtained by respectively describing the corneal confocal image sample along the central line of the hypha and/or the nerve fiber, so that the accuracy of labeling the hypha area and the nerve fiber area is improved.

According to the device for identifying hyphae in a confocal image of cornea provided by the invention, when the pre-segmentation result automatic correction module 30 is used for automatically correcting the pre-segmentation result image to obtain a final segmentation result image, the pre-segmentation result automatic correction module is specifically used for: obtaining each hypha segment and each nerve fiber segment in the pre-segmentation result graph; and calculating preset characteristic indexes of the hypha segments and the nerve fiber segments, comparing the preset characteristic indexes with thresholds of the preset characteristic indexes, and performing class correction on the hypha segments and the nerve fiber segments in the pre-segmentation result graph according to a comparison result to obtain a final segmentation result graph.

According to the device for recognizing the hyphae in the corneal confocal image, the hyphae segmentation and the nerve fiber segmentation in the pre-segmentation result image are obtained, the preset characteristic indexes of the hyphae segmentation and the nerve fiber segmentation are calculated, the preset characteristic indexes are compared with the threshold value of the preset characteristic indexes, and the hyphae segmentation and the nerve fiber segmentation in the pre-segmentation result image are subjected to class correction according to the comparison result to obtain the final segmentation result image, so that the accuracy of the segmentation result is further improved.

According to the device for identifying the hyphae in the corneal confocal image, the preset characteristic indexes comprise at least one of angles among branches at cross points in the segments, lengths of the segments and curvatures of the segments.

According to the device for identifying the hyphae in the corneal confocal image, the accuracy of the segmentation result is further improved by setting the preset characteristic indexes to include at least one of the angle between branches, the length of the segments and the curvature of the segments at the cross points in the segments.

According to the hypha recognition device in the corneal confocal image provided by the invention, the device further comprises a visual display module, wherein the visual display module is used for: and superposing the final segmentation result image and the corneal confocal image to obtain a visual image, and displaying the visual image.

According to the hypha recognition device in the corneal confocal image, the final segmentation result image and the corneal confocal image are overlapped to obtain the visual image, and the visual image is displayed, so that the intuitiveness of the recognition result display is improved.

According to the hypha recognition device in the corneal confocal image provided by the invention, the hypha pre-segmentation module 20 is specifically configured to, when the hypha pre-segmentation module is configured to input the corneal confocal image into a hypha-nerve fiber segmentation model and output a pre-segmentation result map according to the hypha-nerve fiber segmentation model: inputting the corneal confocal image into a skeleton network block of an encoder, and outputting a first characteristic diagram; inputting the first feature map into a large core separable volume block of the encoder, and outputting a second feature map; inputting the second feature map into a self-attention block of the encoder, and outputting a third feature map; and inputting the third feature map into a decoder, and outputting the pre-segmentation result map.

The device for identifying the hyphae in the corneal confocal image provided by the invention has the advantages that the function realization of a hypha-nerve fiber segmentation model is ensured by utilizing the encoder comprising the skeleton network block, the large-kernel separable volume block and the self-attention block to extract the characteristics and utilizing the decoder to output a pre-segmentation result image.

According to the hypha recognition device in the corneal confocal image, provided by the invention, part of deep-layer convolution layers of the skeleton network block are adjusted into deformable convolution layers; and the large kernel separable convolution block divides the input first feature map into two branches, one branch is convolved by a series of 1 × k convolution products and k × 1, the other branch is convolved by a series of k × 1 convolution products and 1 × k, and then the feature maps obtained by the two branches are added point by point to obtain the second feature map.

The device for identifying the hyphae in the corneal confocal image further ensures the function realization of the hypha-nerve fiber by adjusting the skeleton network block into the deformable convolution layer and constructing the large-kernel separable convolution block by utilizing k x 1 convolution and 1 x k convolution.

Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a method of hyphal identification in a confocal image of the cornea, the method comprising: acquiring a corneal confocal image to be identified; inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and judging whether hyphae exist according to the final segmentation result graph.

In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method for identifying hyphae in a confocal image of cornea provided by the above methods, the method comprising: acquiring a corneal confocal image to be identified; inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and judging whether hyphae exist according to the final segmentation result graph.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the above-provided method for hyphal identification in a confocal image of cornea, the method comprising: acquiring a corneal confocal image to be identified; inputting the corneal confocal image into a hypha-nerve fiber segmentation model, and outputting a pre-segmentation result image according to the hypha-nerve fiber segmentation model; the hypha-nerve fiber segmentation model is obtained by taking a corneal confocal image sample as input, taking a labeling result of each pixel point in the corneal confocal image sample belonging to a hypha region, a nerve fiber region or a background region as an output label, and performing machine learning training; automatically correcting the pre-segmentation result graph to obtain a final segmentation result graph; and judging whether hyphae exist according to the final segmentation result graph.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.