阅读说明：本技术 基于增强分辨率的眼底图像的血管管径的测量装置 (Device for measuring blood vessel diameter based on fundus image with enhanced resolution ) 是由 马俊明 王娟 夏斌 于 2020-09-04 设计创作，主要内容包括：本公开描述了一种基于增强分辨率的眼底图像的血管管径的测量装置,包括用于获取眼底图像的获取单元；用于生成血管分割图像的分割单元；基于分辨率增强生成增强血管分割图像的增强单元；获取增强血管分割图像对应的连续血管骨架和测量像素点的管径测量方向的提取单元；基于增强血管分割图像、测量像素点、测量像素点的管径测量方向和预设精度,对增强血管分割图像进行插值操作以生成测量像素点对应的血管轮廓的生成单元；并且基于血管轮廓中的血管像素点的个数、预设倍数和预设精度计算血管管径的计算单元。由此,能够对血管管径进行自动超分辨率测量并具有较高的测量准确度。(The present disclosure describes a measuring device of a blood vessel diameter based on an enhanced resolution fundus image, including an acquisition unit for acquiring a fundus image; a segmentation unit for generating a vessel segmentation image; an enhancement unit generating an enhanced vessel segmentation image based on resolution enhancement; acquiring a continuous blood vessel skeleton corresponding to the enhanced blood vessel segmentation image and an extraction unit for measuring the pipe diameter measuring direction of a measuring pixel point; a generation unit which performs interpolation operation on the enhanced blood vessel segmentation image to generate a blood vessel contour corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, and the caliber measurement direction and preset precision of the measurement pixel point; and a calculating unit for calculating the vessel caliber based on the number of vessel pixel points in the vessel contour, the preset multiple and the preset precision. Therefore, the automatic super-resolution measurement can be performed on the blood vessel diameter, and the measurement accuracy is high.)

1. A measuring device of the blood vessel diameter of a fundus image based on enhanced resolution is characterized by comprising an acquisition unit, a segmentation unit, an enhancement unit, an extraction unit, a generation unit and a calculation unit; the acquisition unit is used for acquiring a fundus image including blood vessels; the segmentation unit generates a blood vessel segmentation image based on the fundus image; the enhancement unit is used for carrying out resolution enhancement on the blood vessel segmentation image according to a preset multiple so as to generate an enhanced blood vessel segmentation image; the extraction unit is used for acquiring a continuous blood vessel skeleton corresponding to the enhanced blood vessel segmentation image and the caliber measurement direction of measurement pixel points, wherein the measurement pixel points are a plurality of pixel points on the continuous blood vessel skeleton, and the caliber measurement direction is perpendicular to the tangent line of the continuous blood vessel skeleton at the measurement pixel points; the generation unit is used for carrying out interpolation operation on the enhanced blood vessel segmentation image to generate a blood vessel profile corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point and preset precision, wherein the preset precision is the step length of interpolation sampling; and the calculating unit calculates the vessel caliber corresponding to the measuring pixel point based on the number of vessel pixel points in the vessel contour corresponding to the measuring pixel point, the preset multiple and the preset precision.

2. The measurement device of claim 1, wherein:

and converting the blood vessel diameter into an actual size by multiplying the blood vessel diameter by the distance between the pixel points, wherein the distance between the pixel points is the actual size represented by the distance between the two pixel points in the fundus image.

3. The measurement device of claim 2, wherein:

the method comprises the steps of inputting an image comprising a scale with scales when acquiring the fundus image, carrying out edge detection on the image comprising the scale with scales to obtain a scale range corresponding to a region with clear scales on the scale, and obtaining the pixel point distance based on the scale range and the number of pixel points between the scale ranges in the fundus image.

4. The measurement device of claim 1, wherein:

the blood vessel diameter l corresponding to the measurement pixel point is

l＝n×s/e，

Wherein n is the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point, s is the preset precision, and e is the preset multiple.

5. The measurement device of claim 4, wherein:

the number n of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point is calculated by the following formula:

n＝card({p：p∈P，f(p)>T})，

wherein P is the measurement pixel point, f (P) is a pixel value corresponding to the measurement pixel point P, T is a preset threshold parameter, P is a pixel point set in a blood vessel contour corresponding to the measurement pixel point P, and card represents the cardinal number of the set.

6. The measurement device of claim 5, wherein:

t is 0.9.

7. The measurement device of claim 1, wherein:

the preset multiple is an integer larger than 1, and the preset precision is a decimal larger than 0 and smaller than 1.

8. The measurement device of claim 1, wherein:

the width of the blood vessel contour is N times of the maximum blood vessel caliber, wherein N is an integer from 2 to 5.

9. The measuring device of claim 1,

and extracting the blood vessel skeleton in the enhanced blood vessel segmentation image and fitting the blood vessel skeleton to obtain the continuous blood vessel skeleton and the pipe diameter measuring direction of the measuring pixel point.

10. The measurement device of claim 1, wherein:

training an artificial neural network to obtain an optimal model and performing blood vessel segmentation on the fundus image by using the optimal model to generate the blood vessel segmentation image.

Technical Field

The present disclosure relates to a blood vessel diameter measuring device based on an enhanced resolution fundus image.

Background

Fundus oculi lesions generally refer to ocular lesions caused by inflammation, tumor, vascular lesions, or the like. Since the ocular fundus diseases not only affect the vision of the patient but also cause blindness in the serious patients, ocular fundus examination should be performed at regular intervals so that the ocular fundus diseases can be detected early and treated in time. Fundus lesions can respond to a variety of signs. Wherein changes in the vessel diameter in the fundus are an important sign of fundus lesions. Therefore, in fundus examination, it is common to obtain a change in the vessel diameter by measuring the vessel diameter in the fundus to identify fundus lesions.

Most of the traditional blood vessel diameter measuring methods (such as a projection method and an ophthalmoscope image measuring method) depend on professional ophthalmologists to measure by means of measuring tools according to experience, although the measuring accuracy is high, the measuring method is long in time consumption, cannot perform batch automatic processing, and is not beneficial to identification of fundus lesions. Therefore, in the field of fundus image analysis, a measurement method which has high accuracy and can automatically measure the vessel diameter has become an important research content. Compared with the traditional pipe diameter measuring method, the fundus image has the advantages of short acquisition time, low acquisition cost and the like, so in recent years, more and more researchers use the fundus image to automatically measure the pipe diameter of the blood vessel. In particular, with the development of computer image processing technology, automatic measurement of blood vessel diameter based on fundus images has been widely studied. For example, Gregoson et al researchers use a blood vessel gray-level contour curve rectangle fitting method to automatically measure the vessel diameter; peter et al performed automatic measurement of vessel diameter by searching for the second derivative zero point of the vessel gray-scale profile curve. However, the accuracy of the automatic measurement of the vessel diameter still needs to be improved.

Disclosure of Invention

The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a method and an apparatus for measuring a blood vessel diameter, which are capable of performing automatic super-resolution measurement of a blood vessel diameter and have a fundus image with high measurement accuracy.

To this end, the present disclosure provides, in a first aspect, a method of measuring a blood vessel diameter of a fundus image, including: acquiring a fundus image including a blood vessel; generating a blood vessel segmentation image based on the fundus image; performing resolution enhancement on the blood vessel segmentation image according to a preset multiple to generate an enhanced blood vessel segmentation image; extracting a blood vessel skeleton in the enhanced blood vessel segmentation image and fitting the blood vessel skeleton to obtain a continuous blood vessel skeleton and a caliber measuring direction of measuring pixel points, wherein the measuring pixel points are a plurality of pixel points on the continuous blood vessel skeleton, and the caliber measuring direction is perpendicular to a tangent line of the continuous blood vessel skeleton at the measuring pixel points; generating a blood vessel contour corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the pipe diameter measurement direction of the measurement pixel point and preset precision; and calculating the vessel caliber corresponding to the measurement pixel point based on the number of the vessel pixel points in the vessel contour corresponding to the measurement pixel point, the preset multiple and the preset precision. In the disclosure, a blood vessel segmentation image is enhanced to generate an enhanced blood vessel segmentation image, a blood vessel skeleton of the enhanced blood vessel segmentation image is extracted and fitted to obtain a continuous blood vessel skeleton and a pipe diameter measuring direction of a measuring pixel point, a blood vessel contour corresponding to the measuring pixel point is generated based on the enhanced blood vessel segmentation image, the measuring pixel point on the continuous blood vessel skeleton, the pipe diameter measuring direction of the measuring pixel point and a preset precision, and a blood vessel pipe diameter corresponding to the measuring pixel point is calculated through the number, preset multiple and preset precision of the blood vessel pixel points in the blood vessel contour corresponding to the measuring pixel point. In this case, the resolution of the blood vessel in the fundus image can be increased, and the blood vessel diameter can be measured with more pixel points. Therefore, automatic super-resolution measurement can be performed on the blood vessel diameter, and the measurement accuracy is improved.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, an interpolation algorithm is used to interpolate the enhanced blood vessel segmentation image to generate the blood vessel contour corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point, and the preset precision. Thus, the enhanced blood vessel image can be interpolated based on the preset precision to acquire the blood vessel contour.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, the blood vessel contours corresponding to the measurement pixel points are arranged side by side in a manner that centers of the blood vessel contours are collinear according to an arrangement order of the measurement pixel points on the continuous blood vessel skeleton to form a straightened blood vessel image. Therefore, the blood vessel contour corresponding to each measurement pixel point can be conveniently acquired.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, the blood vessel diameter l corresponding to the measurement pixel point is equal to/n × s/e, where n is the number of blood vessel pixel points in the blood vessel profile corresponding to the measurement pixel point, s is the preset precision, and e is the preset multiple. From this, the vessel caliber can be calculated.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, the preset multiple is an integer greater than 1, and the preset precision is a decimal greater than 0 and less than 1. Therefore, the resolution ratio of the fundus image blood vessel can be increased, and the measurement accuracy of the subsequent measurement of the blood vessel diameter can be improved.

Further, in the measurement method according to the first aspect of the present disclosure, optionally, the width of the blood vessel profile is N times of the maximum blood vessel diameter, where N is an integer of 2 to 5. This enables the widest blood vessel in the enhanced blood vessel segmentation image to be completely represented by the blood vessel contour.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, the interpolation algorithm is a cubic spline interpolation algorithm. Thus, the enhanced blood vessel segmentation image can be interpolated by a cubic spline interpolation algorithm.

In addition, in the measurement method according to the first aspect of the present disclosure, optionally, the number n of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point is calculated by the following formula: n is card ({ P ∈ P, f (P)) > T }, where P is the measurement pixel, f (P) is a pixel value corresponding to the measurement pixel P, T is a preset threshold parameter, P is a pixel set in the blood vessel contour corresponding to the measurement pixel P, and card represents a cardinal number of the set. Therefore, the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point can be calculated.

A second aspect of the present disclosure provides a measuring apparatus of a blood vessel diameter of a fundus image, including: an acquisition unit for acquiring a fundus image including a blood vessel; a segmentation unit that generates a blood vessel segmentation image based on the fundus image; the enhancement unit is used for carrying out resolution enhancement on the blood vessel segmentation image according to a preset multiple so as to generate an enhanced blood vessel segmentation image; the extraction unit is used for extracting the blood vessel skeleton in the enhanced blood vessel segmentation image and fitting the blood vessel skeleton to obtain a continuous blood vessel skeleton and a pipe diameter measurement direction of measurement pixel points, wherein the measurement pixel points are a plurality of pixel points on the continuous blood vessel skeleton, and the pipe diameter measurement direction is perpendicular to a tangent line of the continuous blood vessel skeleton at the measurement pixel points; the generation unit is used for generating a blood vessel outline corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point and preset precision; and the calculating unit is used for calculating the vessel caliber corresponding to the measuring pixel point based on the number of the vessel pixel points in the vessel contour corresponding to the measuring pixel point, the preset multiple and the preset precision. In the disclosure, the enhancement unit enhances the blood vessel segmentation image generated by the segmentation unit to generate an enhanced blood vessel segmentation image, the extraction unit acquires the continuous blood vessel skeleton and the pipe diameter measurement direction of the measurement pixel point, the generation unit generates a blood vessel contour corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point on the continuous blood vessel skeleton, the pipe diameter measurement direction of the measurement pixel point and the preset precision, and the calculation unit calculates the pipe diameter of the blood vessel corresponding to the measurement pixel point through the number of the blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point, the preset multiple and the preset precision. In this case, the resolution of the blood vessel in the fundus image can be increased, and the blood vessel diameter can be measured with more pixel points. Therefore, automatic super-resolution measurement can be performed on the blood vessel diameter, and the measurement accuracy is improved.

In addition, in the measurement apparatus according to the second aspect of the present disclosure, optionally, an interpolation algorithm is used to interpolate the enhanced blood vessel segmentation image to generate the blood vessel contour corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point, and a preset precision. Thus, the enhanced blood vessel image can be interpolated based on the preset precision to acquire the blood vessel contour.

In addition, in the measurement apparatus according to the second aspect of the present disclosure, optionally, the blood vessel diameter l corresponding to the measurement pixel point is equal to/n × s/e, where n is the number of blood vessel pixel points in the blood vessel profile corresponding to the measurement pixel point, s is the preset precision, and e is the preset multiple. From this, the vessel caliber can be calculated.

According to the present disclosure, a method and an apparatus for measuring a blood vessel diameter of an eye fundus image are provided, which can perform automatic super-resolution measurement of a blood vessel diameter and have high measurement accuracy.

Drawings

The disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

fig. 1 is a schematic diagram showing an application scenario of a method of measuring a blood vessel diameter of a fundus image according to an example of the present disclosure.

Fig. 2 is a flowchart illustrating a method of measuring a blood vessel diameter of a fundus image according to an example of the present disclosure.

Fig. 3 is a schematic diagram illustrating a fundus image according to an example of the present disclosure.

Fig. 4 is a flowchart illustrating a training method for artificial neural network-based vessel segmentation according to an example of the present disclosure.

Fig. 5(a) is a schematic diagram illustrating a training fundus image according to an example of the present disclosure.

Fig. 5(b) is a schematic diagram illustrating a vessel labeling result according to an example of the present disclosure.

Fig. 6 is a schematic diagram illustrating a blood vessel segmentation image according to an example of the present disclosure.

Fig. 7 is a schematic diagram illustrating a partial fundus image according to an example of the present disclosure.

Fig. 8 is a schematic diagram illustrating a local blood vessel segmentation image according to an example of the present disclosure.

Fig. 9 is a schematic diagram illustrating a local vascular skeleton according to an example of the present disclosure.

Fig. 10 is a schematic view showing a pipe diameter measurement direction according to an example of the present disclosure.

Fig. 11 is a flow chart illustrating vessel contouring in accordance with an example of the present disclosure.

Fig. 12 is a schematic diagram illustrating a straightened vessel image according to an example of the present disclosure.

Fig. 13 is a system block diagram showing a measurement apparatus of a blood vessel diameter of a fundus image according to an example of the present disclosure.

Description of the main reference numerals:

1 … application scenario, 11 … operator, 12 … terminal, 13 … acquisition device, 14 … human eye, 15 … server, P1 … fundus image, P2 … training fundus image, P3 … blood vessel labeling result, P4 … blood vessel segmentation image, P5 … local fundus image, P6 … local blood vessel segmentation image, P7 … straightened blood vessel image, L1 … blood vessel skeleton, L2 … continuous blood vessel skeleton, L3 … tangent line, L4 … vessel diameter measurement direction, 2 … measurement device, 210 … acquisition unit, 220 … segmentation unit, 230 … enhancement unit, 240 … extraction unit, 250 … generation unit, 260 … calculation unit, a … blood vessel, B … blood vessel, C … local blood vessel, C' … straightened local blood vessel, D1 … measurement pixel point.

Detailed Description

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same components are denoted by the same reference numerals, and redundant description thereof is omitted. The drawings are schematic and the ratio of the dimensions of the components and the shapes of the components may be different from the actual ones.

It is noted that the terms "comprises," "comprising," and "having," and any variations thereof, in this disclosure, for example, a process, method, system, article, or apparatus that comprises or has a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include or have other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a schematic diagram showing an application scenario of a method of measuring a blood vessel diameter of a fundus image according to an example of the present disclosure.

In some examples, the measurement method of the blood vessel diameter of the fundus image (which may sometimes be simply referred to as a measurement method) to which the present disclosure relates may be applied in the application scenario 1 as shown in fig. 1. In the application scenario 1, the operator 11 can acquire a fundus image of the human eye 14 by controlling the acquisition device 13 connected to the terminal 12. After the acquisition device 13 completes the acquisition of the fundus image, the terminal 12 may submit the fundus image to the server 15 through the computer network. The server 15 can obtain the blood vessel diameter of the fundus image by performing the measurement method according to the example of the present disclosure and return it to the terminal 12. In some examples, terminal 12 may display vessel caliber. In other examples, the vessel diameters may be stored as intermediate results in a memory of the terminal 12 or the server 15.

In some examples, the operator 11 may be a physician with expertise in measuring the vessel caliber of the fundus image. In other examples, the operator 11 may be a general person familiar with how to automatically measure the vessel diameter by operating the terminal 12.

In some examples, the terminal 12 may include, but is not limited to, a laptop, tablet, desktop, or the like. In some examples, the terminal 12 may be a dedicated device for measuring vessel caliber including a processor, memory, display screen and acquisition device 13.

In some examples, the acquisition device 13 may include, but is not limited to, a camera or the like. The camera may be, for example, a color fundus camera. The acquisition device 13 may be connected to the terminal 12 via a serial port or integrated in the terminal 12.

In some examples, the fundus of the human eye 14 refers to the posterior tissue within the eyeball, and may include the inner membrane of the eyeball, the retina, the macula, and blood vessels (the retinal arteriovenous). In some examples, fundus lesions may be identified by monitoring changes in vessel caliber.

In some examples, the server 15 may include one or more processors and one or more memories. Wherein the processor may include a central processing unit, a graphics processing unit, and any other electronic components capable of processing data, capable of executing computer program instructions. The memory may be used to store computer program instructions. In some examples, the measurement methods may be stored in memory and executed by the server 15 in the form of computer program instructions. In some examples, the server 15 may also be a cloud server.

As described above, the server 15 can obtain the blood vessel diameter of the fundus image by performing the measurement method according to the example of the present disclosure. Fig. 2 is a flowchart illustrating a method of measuring a blood vessel diameter of a fundus image according to an example of the present disclosure. Fig. 3 is a schematic diagram illustrating a fundus image according to an example of the present disclosure. Among them, the fundus image P1 is a fundus image captured by a fundus camera.

In some examples, as shown in fig. 2, a method for measuring a vessel diameter of a fundus image according to the present disclosure may include acquiring a fundus image (step S210), performing vessel segmentation (step S220), enhancing resolution (step S230), acquiring a continuous blood vessel skeleton and a vessel diameter measurement direction (step S240), generating a blood vessel profile (step S250), and calculating a vessel diameter (step S260). In this case, the resolution of the blood vessel in the fundus image can be increased, and the blood vessel diameter can be measured with more pixel points. Therefore, automatic super-resolution measurement can be performed on the blood vessel diameter, and the measurement accuracy is improved.

Generally, the width (i.e., the number of pixels) of the blood vessel diameter measurement in the fundus image is a positive integer, and the measurement width may be 1, 3, 6, or 8, for example. The super-resolution technology based on resolution enhancement and preset precision can perform sub-pixel measurement of the vessel diameter, so that the measurement width can be decimal, for example, the measurement width can be 1.23, 3.12, 5.63 or 7.56. Therefore, the vessel diameter can be measured more accurately.

In step S210, a fundus image may be acquired. The fundus image may include blood vessels. In some examples, the fundus image may be a color fundus image. The colored fundus images can clearly present fundus information such as the inner membrane, retina, macula lutea, and blood vessels (retinal artery and vein) of the eyeball. In other examples, the fundus image may be a grayscale image. In some examples, the fundus image may be a fundus image acquired by the acquisition device 13. As an example of the fundus image, fig. 3, for example, shows a fundus image P1 taken by a fundus camera. Among them, the fundus image P1 includes the blood vessel a. In other examples, the fundus image may be an image stored in advance in the terminal 12 or the server 15.

In some examples, in step S210, the fundus image may be pre-processed. In general, since fundus images may have problems such as different image formats and sizes, it is necessary to perform preprocessing on the fundus images to convert the fundus images into images of a fixed standard format. Fixed standard form may mean that the images are of the same format and consistent size. For example, in some examples, the size of the fundus image after preprocessing may be uniform in width of 512 or 1024 pixels.

In step S220, the fundus image acquired in step S210 may be subjected to blood vessel segmentation. In some examples, a vessel segmentation image may be generated based on the fundus image. As described above, the fundus may include the inner membrane of the eyeball, the retina, the macula, and blood vessels (retinal arteriovenous). Therefore, the fundus image of the fundus may include patterns of the inner membrane, retina, macula lutea, and blood vessels (retinal artery and vein) of the eyeball, and performing blood vessel segmentation on the fundus image may identify the blood vessel region, thereby removing the influence of other structures outside the blood vessels.

In general, the blood vessel segmentation method can be classified into an unsupervised learning and a supervised learning method. The unsupervised learning method may first preprocess the fundus image using an image processing technique and then segment the fundus image subjected to the preprocessing by a threshold method.

In some examples, the supervised learning method may extract features of the fundus image to train the classifier by manual or automatic means using the fundus image and its blood vessel labeling results to achieve blood vessel segmentation of the fundus image. For example, the fundus image may be subjected to vessel segmentation by training an optimal model obtained by an artificial neural network.

The following describes a training method for vessel segmentation based on an artificial neural network in detail with reference to the accompanying drawings. Fig. 4 is a flowchart illustrating a training method for artificial neural network-based vessel segmentation according to an example of the present disclosure. Fig. 5(a) is a schematic diagram illustrating a training fundus image according to an example of the present disclosure. Fig. 5(b) is a schematic diagram illustrating a vessel labeling result according to an example of the present disclosure.

As shown in fig. 4, the training method of blood vessel segmentation may include acquiring a training fundus image and a blood vessel labeling result (step S221), preprocessing the training fundus image and the blood vessel labeling result to acquire a preprocessed fundus image and a preprocessed blood vessel labeling result (step S222), and training an artificial neural network based on the preprocessed fundus image and the preprocessed blood vessel labeling result to acquire an optimal model (step S223). In this case, the vessel features may be automatically learned and the vessel segmentation result may be output based on the artificial neural network.

In step S221, a training fundus image and a blood vessel labeling result may be acquired. The training fundus image in step S221 may be a fundus image obtained by photographing the fundus. As an example of the training fundus image, fig. 5(a) shows a training fundus image P2, for example. Among them, the training fundus image P2 includes the blood vessel B. In some examples, the training fundus image may be a plurality of fundus images stored in advance. In some examples, the plurality of training fundus images may constitute a training data set.

Generally, for the supervised learning artificial neural network, a loss function is calculated as a true value based on the vessel labeling result. Therefore, the training fundus image needs to be labeled to acquire a blood vessel labeling result of the training fundus image. The vessel labeling result may include a vessel. In some examples, the training fundus image may be annotated by an experienced physician with an annotation tool to generate vessel annotation results. As an example of the blood vessel labeling result, for example, fig. 5(b) shows a blood vessel labeling result P3 generated after labeling the training fundus image P2. The blood vessel labeling result P3 includes a blood vessel B.

In step S222, the training fundus image and the blood vessel labeling result may be preprocessed to acquire a preprocessed fundus image and a preprocessed blood vessel labeling result. In some examples, the training fundus image may be preprocessed to generate a preprocessed fundus image. In some examples, preprocessing the training fundus image may include cropping, denoising, graying, etc. the training fundus image. This can highlight blood vessels in the training fundus image. In some examples, the training fundus image may be manipulated by rotation, scaling, translation, and so forth. Thus, the data size of the artificial neural network training can be increased. In some examples, the vessel annotation result may be preprocessed to generate a preprocessed vessel annotation result. In some examples, the vessel labeling results may be preprocessed in the same manner as the training fundus image.

In step S223, the artificial neural network may be trained to obtain an optimal model based on the preprocessed fundus image and the preprocessed blood vessel labeling results. In some examples, the artificial neural network may be trained to obtain an optimal model based on the pre-processed fundus image and the pre-processed vessel labeling results. In some examples, the preprocessed vessel labeling results may be used as a true value to calculate a loss function, and the artificial neural network is continuously optimized by the loss function until the convergence of the value (i.e., loss) of the loss function reaches an optimum value.

In some examples, the artificial neural network may be a U-Net network. Specifically, the U-net network is an artificial neural network for image semantic segmentation and comprises a feature extraction part and an up-sampling part. Wherein the feature extraction section may include a plurality of encoding layers, an input of a first encoding layer being a training fundus image. In some examples, the encoding layer may include a series of convolutional layers, batch normalization layers, activation layers, and pooling layers. The up-sampling part may include a plurality of decoding layers. And the last decoding layer (namely an output layer) is used for outputting the probability that each pixel point in the training fundus image belongs to the blood vessel. In this case, a blood vessel segmentation image may be generated based on the probability that each pixel point in the training fundus image belongs to a blood vessel.

Fig. 6 is a schematic diagram illustrating a blood vessel segmentation image according to an example of the present disclosure.

As described above, the artificial neural network may generate a blood vessel segmentation image based on the probability that each pixel point in the training fundus image belongs to a blood vessel. Therefore, the optimal model obtained via training the artificial neural network also has the ability to perform vessel segmentation on the fundus image to generate a vessel segmented image. In some examples, the fundus image acquired in step S210 may be input to an optimal model to perform vessel segmentation on the fundus image, thereby generating a vessel segmentation image. In some examples, the vessel segmentation image may be a binary image, i.e. may comprise only two colors, black and white. As an example of the blood vessel segmentation image, for example, fig. 6 shows a blood vessel segmentation image P4 generated by performing blood vessel segmentation on the fundus image P1. The blood vessel segmentation image P4 includes a blood vessel a.

As described above, the measurement method may include step S230. In step S230, the blood vessel segmentation image acquired in step S220 may be subjected to resolution enhancement. In some examples, the vessel segmentation image may be resolution enhanced by a preset multiple to generate an enhanced vessel segmentation image. For example, in some examples, a 10-fold resolution enhancement may be performed on a vessel segmentation image having a resolution size of 140 × 63 to generate an enhanced vessel segmentation image having a resolution size of 1400 × 630.

In addition, in some examples, the vessel segmentation image may be resolution enhanced using a linear interpolation method. However, examples of the present disclosure are not limited thereto, and in other examples, the vessel segmentation image may be resolution-enhanced using a super-resolution image method based on depth learning.

Additionally, in some examples, the preset multiple may be an integer greater than 1. Therefore, the resolution ratio of the blood vessel in the fundus image can be increased, and the measurement accuracy of the subsequent measurement of the blood vessel diameter can be improved. In some examples, the preset multiple may be 5 to 15. For example, the preset multiple may be 5,10, or 15, etc.

Fig. 7 shows a schematic view of a partial fundus image according to an example of the present disclosure. Here, the local fundus image P5 is a partial view of the fundus image P1, and the local fundus image P5 includes the local blood vessels C. Fig. 8 is a schematic diagram illustrating a local blood vessel segmentation image according to an example of the present disclosure. The local blood vessel segmented image P6 is a local image corresponding to the local fundus image P5 in the blood vessel segmented image P4, and the local blood vessel segmented image P6 includes the local blood vessel C. Fig. 9 is a schematic diagram illustrating a local vascular skeleton according to an example of the present disclosure.

In step S240, the continuous blood vessel skeleton and the caliber measuring direction may be acquired.

In some examples, after the enhanced vessel segmentation image generated in step S230 is acquired, the blood vessel skeleton in the enhanced vessel segmentation image may be extracted. The vascular scaffold may be the midline of a blood vessel. As an example of the blood vessel skeleton, for example, the local blood vessel segmentation image P6 shown in fig. 8 is enhanced to extract the blood vessel skeleton of the local blood vessel C. Fig. 9 shows the blood vessel skeleton L1 of the local blood vessel C corresponding to the enhanced local blood vessel segmentation image P6, and it can be seen from fig. 9 that the blood vessel skeleton L1 is composed of discrete pixel points.

In some examples, the enhanced vessel segmentation image may be refined using a morphological refinement algorithm to extract the vessel skeleton. That is, the width of the blood vessel in the enhanced blood vessel segmentation image is narrowed to the width of one pixel in the blood vessel center direction to form a blood vessel skeleton, and the basic topological structure of the shape of the blood vessel in the enhanced blood vessel segmentation image is kept unchanged. In some examples, the morphology refinement algorithm may include, but is not limited to, a Hilditch refinement algorithm, a Pavlidis refinement algorithm, or a Rosenfeld refinement algorithm, among others. In some examples, the enhanced vessel segmentation image may be subjected to a median filtering operation before being subjected to the vessel skeleton extraction. This enables the removal of possible bifurcations at the end of the vascular skeleton.

As described above, the blood vessel skeleton may be composed of discrete pixel points. Thus, in some examples, the angio-skeleton may be fitted to obtain a continuous angio-skeleton. In some examples, the angioskeleton may be fitted using a least squares cubic spline interpolation algorithm. From this, a continuous vessel skeleton and a fitting equation can be obtained.

Fig. 10 is a schematic view showing a pipe diameter measurement direction according to an example of the present disclosure.

In addition, in some examples, in step S240, the pipe diameter measurement direction of the measurement pixel point may be acquired. The measurement pixel points can be a plurality of pixel points on the continuous blood vessel skeleton. In some examples, the vessel diameter measurement direction may be perpendicular to a tangent of the continuous vessel skeleton at the measurement pixel point. In some examples, the tangent to the measurement pixel may be obtained by the first derivative of the fitting equation described above. As shown in fig. 10, the tangent of the continuous blood vessel skeleton L2 of the local blood vessel C at the measurement pixel point D1 may be the tangent L3. The straight line passing through the measurement pixel point D1 and perpendicular to the tangent line L3 may be the pipe diameter measurement direction L4. In step S250, a vessel contour may be generated. In some examples, the blood vessel contour corresponding to the measurement pixel point may be generated based on the enhanced blood vessel segmentation image obtained in step S230, the measurement pixel point obtained in step S240, the caliber measurement direction of the measurement pixel point obtained in step S240, and a preset precision.

In some examples, an interpolation algorithm may be utilized to interpolate the enhanced blood vessel segmentation image to generate a blood vessel profile corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point, and a preset precision. Thus, the enhanced blood vessel image can be interpolated based on the preset precision to acquire the blood vessel contour. In some examples, the interpolation algorithm may be a cubic spline interpolation algorithm. Thus, the enhanced blood vessel segmentation image can be interpolated by a cubic spline interpolation algorithm.

Additionally, in some examples, the preset precision may be a fraction greater than 0 and less than 1. Therefore, the resolution ratio of the blood vessel in the fundus image can be increased, and the measurement accuracy of the subsequent measurement of the blood vessel diameter can be improved. In some examples, the preset precision may be 0.01 to 0.10. For example, the preset precision may be 0.01,0.05, 0.10, or the like.

The process of generating the blood vessel contour will be described in detail below with reference to the drawings. Fig. 11 is a flow chart illustrating vessel contouring in accordance with an example of the present disclosure.

As shown in fig. 11, the generation process of the blood vessel contour in step S250 may include acquiring a width of the blood vessel contour (step S251), acquiring an interpolation sampling interval (step S252), generating interpolation points (step S253), performing an interpolation operation on the blood vessel segmentation image to determine pixel values of the blood vessel contour (step S254), and generating the blood vessel contour based on the pixel values of the blood vessel contour (step S255).

In step S251, the width of the blood vessel contour may be acquired. In some examples, the width of the vessel profile may be N times the maximum vessel caliber. Wherein N may be an integer between 2 and 5. In some examples, the width of the vessel profile may be 2 times the maximum vessel caliber. In some examples, the maximum vessel caliber may be an estimated maximum vessel caliber. This enables the widest blood vessel in the enhanced blood vessel segmentation image to be completely represented by the blood vessel contour.

In some examples, the estimated maximum vessel caliber may be 2 times the maximum distance among the distances of the pixel points in the continuous vessel skeleton and their nearest non-vessel pixels. Where the distance may be a euclidean distance. Specifically, the Euclidean distance between each pixel point in the continuous blood vessel skeleton and the nearest non-blood vessel pixel is calculated, and 2 times of the maximum Euclidean distance is selected as the estimated maximum blood vessel diameter. From this, the maximum vessel diameter can be roughly estimated. Examples of the disclosure are not limited thereto and in other examples, other ways of estimating the maximum vessel caliber may be used.

In step S252, an interpolated sampling interval may be acquired. In some examples, the interpolated sampling interval may be obtained based on the width of the vessel contour, a preset precision. Specifically, assuming that the width of the blood vessel contour can be represented by wi, the preset accuracy can be represented by s. The value of the interpolation sampling interval may be a value between- (wi-1)/2 and (wi-1)/2, incremented in steps with a preset precision s. For example, the value of the interpolation sampling interval may be- (wi-1)/2, (wi-1)/2+ s, (wi-1)/2+2 xs, or (wi-1)/2, etc.

In step S253, an interpolation point may be generated. In some examples, the interpolation points may be generated based on the interpolation sampling interval, the caliber measurement direction, and the continuous angio-skeleton obtained in step S252. Specifically, assuming that the interpolation sampling interval is denoted by inc, the caliber measurement direction can be denoted by (dx, dy), and the continuous blood vessel skeleton can be denoted by (x, y). The interpolation point can be expressed as (x + dx × inc, y + dy × inc). In this case, the generated interpolation points are distributed along the pipe diameter measurement direction. Therefore, the number of pixels near the pipe diameter measuring direction can be increased, and the measuring accuracy of the pipe diameter of the blood vessel can be improved.

In step S254, an interpolation operation may be performed on the blood vessel segmentation image to determine pixel values of the blood vessel contour. In some examples, an interpolation algorithm may be used to interpolate the blood vessel segmentation image based on the interpolation points obtained in step S253 to determine pixel values of the blood vessel contour. In some examples, the interpolation algorithm may be a cubic spline interpolation algorithm. Thus, the enhanced blood vessel segmentation image can be interpolated by a cubic spline interpolation algorithm.

Fig. 12 is a schematic diagram illustrating a straightened vessel image according to an example of the present disclosure.

In step S255, a blood vessel contour may be generated based on the pixel values of the blood vessel contour. In some examples, the blood vessel profile corresponding to each measurement pixel point may be output based on pixel values of the blood vessel profile. In some examples, the blood vessel profiles corresponding to the measurement pixel points may be arranged side by side in a manner of concentric center lines according to the arrangement order of the measurement pixel points on the continuous blood vessel skeleton to form a straightened blood vessel image. As an example of the straightened blood vessel image, fig. 12 shows a straightened blood vessel image P7. The straightened blood vessel image P7 includes a straightened partial blood vessel C' corresponding to the partial blood vessel C. Therefore, the blood vessel contour corresponding to each measurement pixel point can be conveniently acquired.

As described above, the measurement method includes step S260. In step S260, the vessel caliber may be calculated. In some examples, the vessel caliber corresponding to the measurement pixel point may be calculated based on the number of vessel pixel points in the vessel contour corresponding to the measurement pixel point, a preset multiple, and a preset precision. Therein, the blood vessel profile corresponding to the measurement pixel point can be obtained by step S250.

In some examples, the number n of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point can be calculated by the following formula:

n＝card({p：p∈P，f(p)>T})，

wherein P is a measurement pixel point, f (P) is a pixel value corresponding to the measurement pixel point P, T is a preset threshold parameter, P is a pixel point set in a blood vessel contour corresponding to the measurement pixel point P, and card represents the cardinal number of the set. Therefore, the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point can be calculated. In some examples, T may be 0.9. Therefore, blood vessel pixel points and non-blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel points can be distinguished.

In some examples, the vessel caliber l corresponding to the measurement pixel point may be:

l＝n×s/e，

wherein n is the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point, s is the preset precision, and e is the preset multiple. From this, the vessel caliber can be calculated.

As described above, the preset multiple may be an integer greater than 1. The preset precision may be a decimal greater than 0 and less than 1. In some examples, the vessel diameter of the same segment of blood vessel may be measured by using different preset multiples e and preset precisions s to obtain a measurement result of the vessel diameter with higher accuracy.

Taking the local blood vessel C in the local fundus image P5 shown in fig. 7 as an example, the blood vessel diameter measurement results of the local blood vessel C based on different preset multiples e and preset precision s are compared with the blood vessel diameter measurement results of the local blood vessel C based on artificial labeling, and the comparison results can be seen in table 1. Wherein, the mean value is the average value of the vessel diameters of the local vessels C, and the standard deviation is the standard deviation of the vessel diameters of the local vessels C.

TABLE 1 partial results of comparison of vascular caliber measurements

Measuring parameters	Mean value	Standard deviation of
			Manual labeling	6.4094	0.421
e＝1，s＝1	5.9365	0.7319
			e＝5，s＝0.5	6.4457	0.4737
e＝5，s＝0.1	6.435	0.4723
			e＝5，s＝0.05	6.4341	0.4707
e＝5，s＝0.01	6.4345	0.471
			e＝10，s＝0.5	6.5142	0.4799
e＝10，s＝0.1	6.5078	0.4791
			e＝10，s＝0.05	6.5069	0.4787
e＝10，s＝0.01	6.507	0.4788
			e＝15，s＝0.5	6.5341	0.4757
e＝15，s＝0.1	6.5293	0.4757
			e＝15，s＝0.05	6.5288	0.4754
e＝15，s＝0.01	6.5289	0.4755

As can be seen from the comparison of the blood vessel diameter measurement results in table 1, for the local blood vessel C, the mean value of the artificially labeled blood vessel diameter is 6.4094, and the standard deviation is 0.421. When the super-resolution technology is not used (i.e. e is 1, s is 1), the mean value of the vessel diameter measured by the invention is 5.9365, the standard deviation is 0.7319, and the result is greatly different from the manual labeling result. And when the super resolution technology (i.e. e is equal to {5,10,15} and s is equal to {0.01,0.05,0.1,0.5}) is used, the mean value and the standard deviation of the pipe diameter measurement are close to the manual labeling result. In particular, when e is 5 and s is 0.05, the mean value of the vessel diameters is 6.4341, the standard deviation is 0.4707, and the vessel diameters are equivalent to those of artificially labeled vessels. Therefore, the measurement result of the super-resolution technology is closer to the result of manual marking, and the super-resolution technology effectively improves the accuracy of the blood vessel diameter measurement.

In some examples, the vessel caliber may be turned to actual size. Specifically, an image including a scale may be input at the time of acquiring a fundus image. The scale is provided with scales. In some examples, an image including the scale may be edge-detected based on an image processing algorithm to obtain a region with a sharper scale on the scale, and a scale range corresponding to the region may be obtained, for example, the scale range may be 2um (micrometers) to 100um (micrometers). The pixel point distance can be obtained based on the scale range and the number of the pixel points between the scale range in the eye fundus image. In some examples, the scale range may be used to obtain the actual size, e.g., 98um (microns) divided by the number of pixels in the fundus image between the scale range to obtain the pixel distance. In this case, the vessel diameter can be converted into the actual size by multiplying the vessel diameter by the pixel point distance.

However, examples of the present disclosure are not limited thereto, and in other examples, a fundus image based on DICOM (Digital Imaging and Communications in Medicine) format may acquire a pixel interval (a size of an actual size represented by a distance between two pixel points in the fundus image). In this case, the vessel caliber can be converted to the actual size by multiplying the vessel caliber by the pixel interval. In other examples, the vessel caliber may not be turned to a full size, e.g., the vessel caliber may be taken as an intermediate result.

Hereinafter, a measurement apparatus of a blood vessel diameter of a fundus image according to the present disclosure will be described in detail with reference to fig. 13. The measurement device of the blood vessel diameter of the fundus image in the present disclosure may be simply referred to as "measurement device". The measuring device is used for implementing the measuring method. Fig. 13 is a system block diagram showing a measurement apparatus of a blood vessel diameter of a fundus image according to an example of the present disclosure.

In some examples, as shown in fig. 13, the measurement apparatus 2 may include an acquisition unit 210, a segmentation unit 220, an enhancement unit 230, an extraction unit 240, a generation unit 250, and a calculation unit 260. The acquisition unit 210 may be used to acquire a fundus image. The segmentation unit 220 may be used to perform blood vessel segmentation on the fundus image. The enhancement unit 230 may be used to perform resolution enhancement on the vessel segmentation image. The extraction unit 240 may be used to acquire a continuous blood vessel skeleton and a vessel diameter measurement direction. The generation unit 250 may be used to generate a vessel contour. The calculation unit 260 may be used to calculate the vessel caliber. In this case, the resolution of the blood vessel in the fundus image can be increased, and the blood vessel diameter can be measured with more pixel points. Therefore, automatic super-resolution measurement can be performed on the blood vessel diameter, and the measurement accuracy is improved.

In some examples, the acquisition unit 210 may be used to acquire a fundus image. The fundus image may include blood vessels. In some examples, the fundus image may be a color fundus image. In some examples, the fundus image may be pre-processed. For details, refer to step S210, which is not described herein again.

In some examples, the segmentation unit 220 may be used to perform blood vessel segmentation on the fundus image acquired by the acquisition unit 210. In some examples, a vessel segmentation image may be generated based on the fundus image. In some examples, the fundus image may be vessel segmented by an artificial neural network. In this case, the vessel features may be automatically learned and the vessel segmentation result may be output based on the artificial neural network. For details, refer to step S220, which is not described herein again.

In some examples, the enhancement unit 230 may be used to perform resolution enhancement on the vessel segmentation image. In some examples, the vessel segmentation image may be resolution enhanced by a preset multiple to generate an enhanced vessel segmentation image. In some examples, the vessel segmentation image may be resolution enhanced using a linear interpolation method. In some examples, the preset multiple may be set to an integer greater than 1. Therefore, the resolution ratio of the blood vessel in the fundus image can be increased, and the measurement accuracy of the subsequent measurement of the blood vessel diameter can be improved. For details, refer to step S230, which is not described herein.

In some examples, the extraction unit 240 may be used to acquire a continuous vessel skeleton and vessel diameter measurement direction. In some examples, after the enhanced vessel segmentation image generated by the enhancement unit 230, the vessel skeleton in the enhanced vessel segmentation image may be extracted. The vascular scaffold may be the midline of a blood vessel. In some examples, the enhanced vessel segmentation image may be refined using a morphological refinement algorithm to extract the vessel skeleton. In some examples, the enhanced vessel segmentation image may be subjected to a median filtering operation before being subjected to the vessel skeleton extraction. This enables the removal of possible bifurcations at the end of the vascular skeleton. In some examples, the vascular skeleton may be fitted to obtain a continuous vascular skeleton. In some examples, the angioskeleton may be fitted using a least squares cubic spline interpolation algorithm. From this, a continuous vessel skeleton and a fitting equation can be obtained. In some examples, the extraction unit 240 may be configured to obtain a pipe diameter measurement direction of the measurement pixel. The measurement pixel points can be a plurality of pixel points on the continuous blood vessel skeleton. In some examples, the vessel diameter measurement direction may be perpendicular to a tangent of the continuous vessel skeleton at the measurement pixel point. For details, refer to step S240, which is not described herein.

In some examples, the generation unit 250 may be used to generate a vessel contour. In some examples, the blood vessel contour corresponding to the measurement pixel point may be generated based on the enhanced blood vessel segmentation image obtained by the enhancement unit S230, the measurement pixel point obtained by the extraction unit 240, the caliber measurement direction of the measurement pixel point obtained by the extraction unit 240, and a preset precision. In some examples, an interpolation algorithm may be utilized to interpolate the enhanced blood vessel segmentation image to generate a blood vessel profile corresponding to the measurement pixel point based on the enhanced blood vessel segmentation image, the measurement pixel point, the caliber measurement direction of the measurement pixel point, and a preset precision. Thus, the enhanced blood vessel segmentation image can be subjected to interpolation operation based on preset precision. In some examples, the interpolation algorithm may be a cubic spline interpolation algorithm. Thus, the enhanced blood vessel segmentation image can be interpolated by a cubic spline interpolation algorithm. For details, refer to step S250, which is not described herein.

In some examples, in the generating unit 250, the preset precision may be a fraction greater than 0 and less than 1. Therefore, the resolution ratio of the blood vessel in the fundus image can be increased, and the measurement accuracy of the subsequent measurement of the blood vessel diameter can be improved. In some examples, the width of the vessel profile may be N times the maximum vessel caliber, where N may be an integer from 2 to 5. In some examples, the blood vessel profiles corresponding to the measurement pixel points may be arranged side by side in a manner of concentric lines to form a straightened blood vessel image according to the arrangement sequence of the measurement pixel points on the continuous blood vessel skeleton. For details, refer to step S250, which is not described herein.

In some examples, the calculation unit 260 may be used to calculate the vessel caliber. In some examples, the vessel caliber corresponding to the measurement pixel point may be calculated based on the number of vessel pixel points in the vessel contour corresponding to the measurement pixel point, a preset multiple, and a preset precision. In some examples, the number n of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point can be calculated by the following formula:

n＝card({p：p∈P，f(p)>T})，

wherein P is a measurement pixel point, f (P) is a pixel value corresponding to the measurement pixel point P, T is a preset threshold parameter, P is a pixel point set in a blood vessel contour corresponding to the measurement pixel point P, and card represents the cardinal number of the set. Therefore, the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point can be calculated. In some examples, T may be 0.9. Therefore, blood vessel pixel points and non-blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel points can be distinguished. In some examples, the vessel caliber l corresponding to the measurement pixel point may be:

l＝n×s/e，

wherein n is the number of blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point, s is the preset precision, and e is the preset multiple. From this, the vessel caliber can be calculated. For details, refer to step S260, which is not described herein again.

In the disclosure, a blood vessel segmentation image is enhanced to generate an enhanced blood vessel segmentation image, a blood vessel skeleton of the enhanced blood vessel segmentation image is extracted and fitted to obtain a continuous blood vessel skeleton and a pipe diameter measurement direction, a blood vessel contour corresponding to a measurement pixel point is generated based on the enhanced blood vessel segmentation image, the measurement pixel point on the continuous blood vessel skeleton, the pipe diameter measurement direction of the measurement pixel point and a preset precision, and a blood vessel pipe diameter corresponding to the measurement pixel point is calculated through the number, preset multiple and preset precision of the blood vessel pixel points in the blood vessel contour corresponding to the measurement pixel point. In this case, the resolution of the blood vessel in the fundus image can be increased, and the blood vessel diameter can be measured with more pixel points. Therefore, the measurement accuracy of the vessel diameter can be improved.

While the present disclosure has been described in detail in connection with the drawings and examples, it should be understood that the above description is not intended to limit the disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from the true spirit and scope of the disclosure, which fall within the scope of the disclosure.