阅读说明：本技术 一种异常细胞图像特征的检索方法、装置、设备和介质 (Retrieval method, device, equipment and medium for abnormal cell image features ) 是由 初晓 韩英男 刘浩 王坚 平波 喻林 于 2021-09-16 设计创作，主要内容包括：本发明提出一种异常细胞图像特征的检索方法、装置、设备和介质,涉及图像处理应用领域,该方法包括：构建用于获取输入图像的二进制编码特征的特征编码网络；获取样本数据集,所述样本数据集包括细胞样本图像；将所述样本数据集输入至特征编码网络,获取样本编码特征结果,并将其作为异常细胞图像特征,并根据异常细胞图像特征与异常描述文本的关联关系,建立表征异常细胞图像特征的标准特征库；将待检索样本图像输入所述特征编码网络,获取待检索编码特征结果,进而通过所述标准特征库,获取与所述待检索编码特征结果匹配的异常描述文本,完成异常细胞图像特征的检索；本发明为细胞病理性诊断提供可靠的参考数据,同时保障特征检索的效率。(The invention provides a method, a device, equipment and a medium for searching abnormal cell image characteristics, which relate to the field of image processing application, and the method comprises the following steps: constructing a feature coding network for acquiring binary coding features of an input image; acquiring a sample data set, wherein the sample data set comprises a cell sample image; inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and an abnormal description text; inputting a sample image to be retrieved into the feature coding network, acquiring a feature result of the code to be retrieved, further acquiring an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library, and completing retrieval of the abnormal cell image features; the invention provides reliable reference data for the pathological diagnosis of cells and ensures the efficiency of characteristic retrieval.)

1. A method for retrieving an image feature of an abnormal cell, comprising:

constructing a feature coding network for acquiring binary coding features of an input image;

acquiring a sample data set, wherein the sample data set comprises a cell sample image;

inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and an abnormal description text;

and inputting the sample image to be retrieved into the feature coding network to obtain a feature result of the code to be retrieved, and further obtaining an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library to complete the retrieval of the abnormal cell image features.

2. The method for retrieving the abnormal cell image feature according to claim 1, wherein the step of inputting the sample data set to a feature coding network, acquiring a sample coding feature result, and using the sample coding feature result as the abnormal cell image feature comprises:

sorting the cell sample images, classifying and labeling the cell sample images according to a preset disease classification standard, and creating a sample data set;

selecting a specified number of samples from the sample data set to train the feature coding network;

and carrying out classification detection on all sample images in the sample data set according to the trained feature coding network to obtain corresponding abnormal cell image features.

3. The method for retrieving the abnormal cell image feature according to claim 1, wherein before inputting the sample data set to a feature coding network, obtaining a sample coding feature result and using the sample coding feature result as the abnormal cell image feature, the method further comprises:

creating an abnormal description text of each feature according to a specific disease classification standard;

matching description texts of corresponding categories according to the labeling information of the cell sample images to obtain abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories;

and acquiring the sample image and the corresponding abnormal description text, outputting the sample image and the corresponding abnormal description text to a specific target object for verification, and judging whether the sample image is correctly associated with the abnormal description text.

4. The method for retrieving the abnormal cell image features according to claim 1, wherein the feature coding network comprises: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit;

the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel;

the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

5. The method for retrieving the abnormal cell image features according to claim 4, wherein the feature extraction is performed by using each feature extraction channel of the multi-scale attention unit, and the method comprises the following steps:

performing convolution on the input image through the convolution layers of the channels respectively to obtain a feature map output by each convolution layer;

setting a channel attention unit, acquiring the evaluation score of each channel feature map as channel attention, introducing the evaluation score into the output of a corresponding channel, and acquiring an attention vector of a corresponding feature extraction channel;

and weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-connection layer of the coding unit.

6. The abnormal cell image feature retrieval method according to claim 3, wherein the channel attention unit comprises a global average pooling layer, a first full-link layer, a first activation function, a second full-link layer and a second activation function which are connected in sequence;

inputting the feature map obtained by the feature extraction channel into the global average pooling layer to obtain a scalar of the corresponding channel; and after the scalar sequentially passes through a first full connection layer, a first activation function, a second full connection layer and a second activation function, obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel, wherein the first activation function adopts a ReLU function, and the second activation function adopts a Sigmoid function.

7. The abnormal cell image feature retrieval method according to claim 4, wherein the fully-connected layer of the coding unit is two layers, and includes a third fully-connected layer and a fourth fully-connected layer, and the third fully-connected layer outputs the acquired feature map to the fourth fully-connected layer and the hash layer, respectively; and the fourth full connection layer outputs the acquired feature map to the hash layer.

8. An apparatus for retrieving an image feature of an abnormal cell, comprising:

the network construction module is used for constructing a feature coding network for acquiring binary coding features of an input image;

the sample data acquisition module is used for acquiring a sample data set, wherein the sample data set comprises a cell sample image;

the standard feature library creating module is used for inputting the sample data set into a feature coding network, acquiring a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and creating a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and the abnormal description text;

and the abnormal feature retrieval module is used for inputting the sample image to be retrieved into the feature coding network to obtain the coding feature result to be retrieved, further obtaining an abnormal description text matched with the coding feature result to be retrieved through the standard feature library, and completing retrieval of the abnormal cell image features.

9. A computer device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.

Technical Field

The invention relates to the field of image processing application, in particular to a method, a device, equipment and a medium for abnormal cell image characteristics.

Background

Cervical cancer is one of the malignancies that poses a serious health hazard to women, with the incidence second among women's malignancies. There are 50 million cases of onset and 27.4 cases of death worldwide each year, with 85% of cervical cancer cases occurring in mid-to-low income countries with low rates of census. Cervical cancer is the only cancer that can be detected and cured early at present, so early screening is critical to the treatment of cervical cancer.

The cervical liquid-based cell inspection method is the most common cervical cancer screening method at present, but the general investigation rate of the cervical cancer is only 1 percent in China due to the lack of pathologists and cytological detection equipment. The traditional cell pathological diagnosis is mostly judged by depending on the experience of professional doctors, and the accuracy is difficult to guarantee.

Disclosure of Invention

In view of the problems in the prior art, the invention provides a method, a device, equipment and a medium for searching abnormal cell image features, and mainly solves the problems that the conventional method relies on manual experience to perform pathological judgment, and the clinical diagnosis processing efficiency is low and the accuracy is poor.

In order to achieve the above and other objects, the present invention adopts the following technical solutions.

A method for searching abnormal cell image features comprises the following steps:

constructing a feature coding network for acquiring binary coding features of an input image;

acquiring a sample data set, wherein the sample data set comprises a cell sample image;

inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and an abnormal description text;

and inputting the sample image to be retrieved into the feature coding network to obtain a feature result of the code to be retrieved, and further obtaining an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library to complete the retrieval of the abnormal cell image features.

Optionally, inputting the sample data set to a feature coding network, obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature, including:

sorting cell sample images corresponding to a specific disease, classifying and labeling the cell sample images according to a preset disease classification standard, and creating a sample data set;

selecting a specified number of samples from the sample data set to train the feature coding network;

and carrying out classification detection on all sample images in the sample data set according to the trained feature coding network to obtain corresponding abnormal cell image features.

Optionally, before inputting the sample data set to a feature coding network, obtaining a sample coding feature result and using the sample coding feature result as an abnormal cell image feature, the method further includes:

creating an abnormal description text of each feature according to a specific disease classification standard;

matching description texts of corresponding categories according to the labeling information of the cell sample images to obtain abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories;

and acquiring the sample image and the corresponding abnormal description text, outputting the sample image and the corresponding abnormal description text to a specific target object for verification, and judging whether the sample image is correctly associated with the abnormal description text.

Optionally, the feature encoding network comprises: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit;

the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel;

the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

Optionally, the feature extraction is performed by using each feature extraction channel of the multi-scale attention unit, and the method includes:

performing convolution on the input image through the convolution layers of the channels respectively to obtain a feature map output by each convolution layer;

setting a channel attention unit, acquiring the evaluation score of each channel feature map as channel attention, introducing the evaluation score into the output of a corresponding channel, and acquiring an attention vector of a corresponding feature extraction channel;

and weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-connection layer of the coding unit.

Optionally, the channel attention unit includes a global average pooling layer, a first fully-connected layer, a first activation function, a second fully-connected layer, and a second activation function, which are connected in sequence;

inputting the feature map obtained by the feature extraction channel into the global average pooling layer to obtain a scalar of the corresponding channel; and after the scalar sequentially passes through a first full connection layer, a first activation function, a second full connection layer and a second activation function, obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel, wherein the first activation function adopts a ReLU function, and the second activation function adopts a Sigmoid function.

Optionally, the full-link layer of the coding unit is two layers, and includes a third full-link layer and a fourth full-link layer, and the third full-link layer outputs the obtained feature map to the fourth full-link layer and the hash layer, respectively; and the fourth full connection layer outputs the acquired feature map to the hash layer.

An abnormal cell image feature retrieval device, comprising:

the network construction module is used for constructing a feature coding network for acquiring binary coding features of an input image;

the sample data acquisition module is used for acquiring a sample data set, wherein the sample data set comprises a cell sample image;

the standard feature library creating module is used for inputting the sample data set into a feature coding network, acquiring a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and creating a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and the abnormal description text;

and the abnormal feature retrieval module is used for inputting the sample image to be retrieved into the feature coding network to obtain the coding feature result to be retrieved, further obtaining an abnormal description text matched with the coding feature result to be retrieved through the standard feature library, and completing retrieval of the abnormal cell image features.

A computer device, comprising: the abnormal cell image feature retrieval method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the steps of the abnormal cell image feature retrieval method when executing the computer program.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for retrieving an image feature of abnormal cells.

As described above, the dynamic contract signing method, apparatus, device and medium of the present invention have the following advantageous effects.

The historical cell images are classified, a standard sample feature library is created, the cell image features are coded through a deep neural network, retrieval matching is directly carried out according to the cell feature codes during searching, and retrieval efficiency can be greatly improved; the accuracy of feature extraction and feature comparison is improved by constructing a deep neural network; the doctor or the patient can directly use the current image to be retrieved as input, quickly compare the current image to be retrieved with the features in the feature library to obtain the matched features, obtain related description information such as symptom description, medication suggestion, diet contraindication and the like according to the features, quickly obtain reference data, improve the efficiency of obtaining information by the doctor or the patient and provide available reference data for inexperienced doctors.

Drawings

Fig. 1 is a flowchart illustrating a method for retrieving abnormal cell image features according to an embodiment of the present invention.

Fig. 2 is a schematic flow chart of multi-scale feature extraction according to an embodiment of the present invention.

Fig. 3 is a schematic flow chart illustrating the process of obtaining abnormal cell image features of a cell sample image according to an embodiment of the present invention.

Fig. 4 is a flow chart illustrating the association between a cell sample image and an anomaly description text according to an embodiment of the present invention.

FIG. 5 is a block diagram of an apparatus for retrieving abnormal cell image features according to an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

Referring to fig. 1, the present invention provides a method for retrieving an abnormal cell image feature, comprising the following steps:

step S1, constructing a feature coding network for acquiring binary coding features of the input image;

step S2, obtaining a sample data set, wherein the sample data set comprises a cell sample image;

step S3, inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and the abnormal description text;

and step S4, inputting the sample image to be retrieved into the feature coding network, obtaining the feature result of the code to be retrieved, further obtaining the abnormal description text matched with the feature result of the code to be retrieved through the standard feature library, and completing the retrieval of the abnormal cell image feature.

The following describes the workflow steps of the abnormal cell image feature search method according to the present invention with reference to specific embodiments.

In step S1, a feature encoding network for acquiring binary-encoded features of an input image is constructed.

In one embodiment, a feature encoding network comprises: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit; the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel; the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

Specifically, taking cervical cell classification as an example, in order to better adapt to the characteristic features of cervical cells different from other target detection tasks, an attention mechanism-added efficient Pyramid feature extraction network can be adopted to improve the network, a conventional convolution kernel is replaced by an epsa (efficient Pyramid spread attribute) module as a multi-scale feature extraction unit, multi-scale spatial information and a cross-channel attention mechanism are integrated, feature information can be extracted at a finer granularity level, the importance of each channel is selectively weighed, and therefore a richer feature map refined by the multi-scale feature information is obtained and used as output. And finally, passing through a full-connection layer neural network, wherein the full-connection layer neural network firstly converts the output of the feature extraction network into a multidimensional hash layer for outputting the features of the image, then uses an activation function for image coding, restrains the value in the hash layer in (0,1), and then performs binary coding on the data in the hash layer through a threshold function, wherein the binary coding is 1 when the value is larger than the threshold value, and is 0 when the value is not larger than the threshold value. For example, if the hash layer includes 48 output neurons, 48-bit binary hash codes are generated after passing through the hash layer, so as to extract image features, and the image features are stored in the database.

Referring to fig. 2, in an embodiment, the feature extraction performed by each feature extraction channel of the multi-scale attention unit includes the following steps:

s101, performing convolution on an input image through convolution layers of a plurality of channels to obtain a feature map output by each convolution layer;

s102, setting a channel attention unit, acquiring an evaluation score of each channel feature map as channel attention, introducing the evaluation score into corresponding channel output, and acquiring an attention vector of a corresponding feature extraction channel;

s103, weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-link layer of the coding unit.

Specifically, the input image may be split into a plurality of portions, each portion corresponds to one channel, and feature extraction is performed on each portion through the convolution layer corresponding to the channel, so as to obtain features of different scales. And then carrying out feature splicing on the feature graphs of different scales corresponding to the channels through a splicing layer (concat layer). Prior to feature stitching, the attention vector for each channel is calculated.

In an embodiment, the channel attention unit comprises a global average pooling layer, a first fully connected layer, a first activation function, a second fully connected layer, and a second activation function connected in sequence. And obtaining a scalar of the corresponding channel after the feature diagram of each channel passes through the global average pooling layer, and obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel after passing through the first full-connection layer, the first activation function, the second full-connection layer and the second activation function. And then weighting according to the attention weight of each channel (namely multiplying each element corresponding to the channel by the attention weight), and taking the weighted feature map as the calibration feature map of the channel. In the first step, the overall average pooling of the number of HxW of each channel is performed to obtain a scalar, which is called as Squeeze, then a weight value between 0 and 1 is obtained through two full connection layers, and each element of each original HxW is multiplied by the weight of the corresponding channel to obtain a new feature map. The first activation function may employ a ReLU function and the second activation function may employ a Sigmoid function. Because the importance of each channel is different, the channel attention unit is mainly used for autonomously learning the correlation between the channels and screening out the attention weight aiming at the channels.

For CNN networks, the core computation is a convolution operator, which learns a new feature map from an input feature map through a convolution kernel. Essentially, convolution is the feature fusion of a local region, which includes feature fusion spatially (H and W dimensions) and inter-channel (C dimension). The receptive field of the ordinary convolutional neural network is not large, and more channel characteristics can be designed to increase the receptive field, but the calculation amount is greatly increased. Therefore, in order to spatially fuse more features, or to extract multi-scale spatial information. Many different approaches such as the multi-branch structure of the inclusion network have also been proposed by those skilled in the art. For feature fusion of channel dimensions, the convolution operation basically fuses all channels of the input feature map by default. The channel attention unit in this embodiment focuses on the relationship between channels, and it is expected that the model can automatically learn the importance of different channel features. Therefore, by processing the feature map obtained by the convolution, a scalar corresponding to the channel is obtained as an evaluation score of each channel, and then the scores are applied to the corresponding channels, respectively, to obtain a corrected feature map.

In one embodiment, after feature correction and feature stitching, the feature map is input into a fully-connected layer neural network. The full-connection layer neural network comprises a full-connection layer and a hash layer. The dimension of the hash layer can be adjusted according to application requirements, and the dimension of the hash layer can be set to 48 as an example. Each neuron output in the full-connection layer is connected with an activation function, the activation function can adopt a sigmoid function, neurons in the hash layer are activated through the activation function, and the characteristic value output by the full-connection layer is restricted to be between 0 and 1. And the Hash layer carries out threshold judgment through a threshold function, wherein the output of the Hash layer is 1 when the Hash layer is larger than the threshold, and the output of the Hash layer is 0 when the Hash layer is smaller than the threshold. A 48-bit binary encoding of the cellular anomaly features is generated by a hash layer. The number of bits of the binary code can be adjusted by adjusting the dimension of the hash layer.

In an embodiment, the activation function of the hash layer may also adopt a tanh function. If the sign function is directly applied to the output of the fully connected layer, the output can be strictly limited to a binary output, but it is not differentiable, and therefore it is difficult to update the gradient by a back-propagation gradient, which is not favorable for optimization of the model. If the ReLU function is chosen, a discrete optimization problem that is difficult to directly optimize is faced when minimizing the loss function. In order to use continuous output as relaxation of the hash code, sigmoid or tanh can be used as an activation function, but the output value of the sigmoid function is not 0 as a central value, and the output range is always positive in the interval of (0,1), so that a signal with a non-0-mean value is input to a neuron of a next layer, and the influence on the gradient is generated. In the training process, all the gradient values of the parameters are positive numbers or negative numbers, so that the parameters are updated in the positive direction or the negative direction in the back propagation process, the convergence is slow, and the optimal value is not easy to reach. And the tanh function is 0 mean value, so that the problem that the sigmoid function is not 0 mean value is solved, and the training efficiency is improved.

In one embodiment, the fully-connected layer may be two fully-connected layers, the third fully-connected layer and the fourth fully-connected layer are connected in sequence, and the output of each fully-connected layer is connected to the hash layer. The Hash layer can be connected with an output layer to carry out back propagation to update network parameters through a loss function of the output layer, and model training is carried out. And finishing model training, and acquiring the output of the Hash layer as the abnormal cell image characteristics of the corresponding input image. The characteristics output by the fourth fully-connected layer can be used as the visual characteristics of the images, but the characteristics are excessively dependent on classes and have strong invariance, so that the subtle semantic distinction between the captured images is not facilitated, the hash layer is connected to the two fully-connected layers, the existing information loss condition can be reduced, the hash code can better utilize diversified information biased to the visual appearance, and therefore the input of the hash layer is divided into two parts, one part is from the second fully-connected layer, and the other part is from the first fully-connected layer.

In the image feature extraction link, the traditional mode is based on artificially extracted image bottom layer features, but the bottom layer visual descriptor has the problem of insufficient expression capability. Compared with the manual feature extraction, the deep convolution neural network can obtain the internal features of the image more, and the extracted features are more accurate. In the matching process, the traditional linear scanning is realized by carrying out similarity calculation with each data point in a database one by one. As the database scale becomes larger and the feature vector dimension becomes higher, the calculation amount becomes huge, and the search cost also gradually increases. The image retrieval based on the Hash coding can reduce the space complexity and time complexity of searching, the high-dimensional feature representation of the image is mapped into binary coding, the dimension reduction of data is realized, and the measurement operation can be carried out in a low-dimensional space.

In step S2 and step 3, a sample data set is acquired, the sample data set including a cell sample image; and inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and the abnormal description text.

Referring to fig. 3, in an embodiment, inputting the sample data set to a feature coding network, obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature includes the following steps:

step, 201: sorting cell sample images corresponding to a specific disease, classifying and labeling the cell sample images according to a preset disease classification standard, and creating a sample data set;

step S202: selecting a specified number of samples from the sample data set to train the feature coding network;

step S203: and carrying out classification detection on all sample images in the sample data set according to the trained feature coding network to obtain corresponding abnormal cell image features.

Specifically, taking the image of the abnormal cervical cell sample as an example, the image of the abnormal cervical cell sample can be obtained by imaging through a microscope or a digital medical record scanner. The acquired sample images may include the following categories:

a. normal squamous epithelial cells and abnormally altered squamous epithelial cells (atypical squamous epithelial cells of no clear diagnostic significance (ASC-US), low-grade squamous intraepithelial lesions (LSIL), high-grade squamous intraepithelial lesions (HSIL))

b. Normal glandular epithelial cells and abnormal glandular epithelial cells (atypical glandular epithelial cells (AGC) without definitive diagnostic significance, cervical adenocarcinoma)

c. Microorganism (herpes, actinomycete, trichomonad)

Furthermore, abnormal regions in the sample image can be labeled by expert doctors, the abnormal categories corresponding to the special abnormal regions, such as low-grade squamous intraepithelial lesions, high-grade squamous intraepithelial lesions and the like,

and acquiring the labeled sample data set.

Referring to fig. 4, in an embodiment, before inputting the sample data set to a feature coding network, and obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature, the method further includes the following steps:

step S204, establishing an abnormal description text of each characteristic according to a specific disease classification standard;

step S205, matching description texts of corresponding categories according to the labeling information of the cell sample images, and obtaining abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories;

step S206, the sample image and the corresponding abnormal description text are obtained and output to a specific target object for verification, and whether the sample image and the abnormal description text are correctly associated is judged.

Specifically, taking the TBS (the Bethesda system) diagnosis as an example, TBS diagnosis is a descriptive diagnosis, primarily describing the condition of cervical cells. Two major terms are mentioned: low-grade squamous intraepithelial lesions (LSIL) and high-grade squamous intraepithelial lesions (HSIL)). Corresponding descriptive text may be generated for sample images of both types of lesions. The anomaly descriptive text content may include diagnostic criteria, treatment methods, precautions, and the like. Illustratively, the low-grade squamous epithelial lesion diagnostic criteria include: cells in single or sheet arrangement, cells that are "mature" or are surface-type cytoplasm, and clear cytoplasmic boundaries, etc. And generating a description text corresponding to each diagnosis segment standard according to the data in the TBS diagnosis standard. Different disease types may correspond to different diagnostic criteria, and the TBS diagnostic criteria herein should not be considered as limiting the present embodiment. The corresponding relationship between the diagnostic standard and the abnormal description text can be set according to the actual application requirements, and is not limited herein. And inputting the abnormal description text into a database to obtain a description text database.

After the sample image is obtained, the sample image can be output to an application end corresponding to the expert group, and the expert group labels the characteristic category in the sample image to obtain labeling information. Further, the matched abnormal description text can be obtained from the description text database according to the labeling information of each sample image. Specifically, the euclidean distance or the canonical form distance between the labeled feature type and the anomaly description text can be calculated to obtain the similarity between the two. And if the similarity of the two images reaches a preset similarity threshold, associating the sample image with the corresponding abnormal description text. And outputting the sample image and the associated abnormal description text to an application end corresponding to the expert group, and correcting the abnormal description text by the expert group.

After the sample image is subjected to feature extraction through the feature coding network, the incidence relation between the abnormal cell image features corresponding to the sample image and the abnormal description text can be established according to the incidence relation between the sample image and the abnormal description text.

And inputting the labeled sample data and the feature coding network constructed through the steps, and performing model training to obtain an abnormal cell image feature acquisition model.

In S3, inputting the sample image to be retrieved into the feature coding network to obtain the feature to be retrieved;

in an embodiment, the sample to be retrieved may be input into the abnormal cell image feature obtaining model obtained through model training in step S2, so as to obtain the feature to be retrieved.

After the standard feature library corresponding to the sample image is obtained in step S2, when a intern or other inexperienced person cannot accurately diagnose the currently acquired cell image, the acquired image may be entered into the system to perform image feature retrieval, so as to obtain reference information, so as to assist the corresponding person in cell abnormality determination. Specifically, the currently acquired cell image can be used as a sample image to be retrieved and input into an abnormal cell image feature acquisition model, and the abnormal cell image features corresponding to the sample image to be retrieved are converted into binary hash codes with corresponding digits through model feature extraction to be used as the features to be retrieved.

In step S4, the matched feature and the associated description text are obtained from the standard feature library according to the feature to be retrieved.

When the matching features are retrieved from the standard feature library, binary hash code comparison can be performed according to the features to be retrieved obtained in step S2, and the hamming distance between two binary hash codes is calculated as the similarity between the two binary hash codes. And acquiring the abnormal cell image features with the highest similarity, or acquiring the first K abnormal cell image features reaching the similarity threshold value and outputting the first K abnormal cell image features as matching features. And meanwhile, outputting a corresponding description text for a searcher to refer to according to the incidence relation between the abnormal cell image characteristics and the abnormal description text.

In an embodiment, after the matched abnormal cell image features are obtained, key information in the associated abnormal description text, such as category information included in the abnormal description text, may be further extracted, and the corresponding key features are extracted according to each category information and output to the retrieving party as category identifiers. And the retrieval party can click the corresponding key characteristics to trigger the jump connection of the secondary interface and acquire the secondary interface. And displaying the description text corresponding to the key information in the secondary interface. When the description text with larger information amount is processed, a searching party is not required to screen the information, corresponding text data is automatically classified and output, and the use experience is enhanced.

And marking the sample image according to a specific diagnosis standard, establishing a standard feature library, and associating feature description texts, so that a cytopathologist can quickly acquire accurate diagnosis reference data through a search platform. By adopting a network of a high-efficiency pyramid segmentation feature extraction module with an attention mechanism, multi-scale spatial information and a cross-channel attention mechanism are integrated, so that pathological image features of cervical cancer cells are better fitted, detailed information of the cervical cancer cells is better extracted, and the detection accuracy is improved. Meanwhile, binary hash code coding is adopted, and the retrieval efficiency is improved. For doctors with insufficient clinical diagnosis experience, similar typical examples are searched in the pathology database, and description and judgment of experts can be referred to assist the doctors in carrying out pathology diagnosis, so that comprehensive analysis can be carried out by combining historical pathology library diagnosis results, and clinical efficiency and accuracy are improved.

In one embodiment, as shown in fig. 5, an apparatus for retrieving an image feature of an abnormal cell is provided, the apparatus comprising: a network construction module 10, configured to construct a feature coding network for obtaining binary coding features of an input image; the sample data acquisition module 11 is configured to acquire a sample data set, where the sample data set includes a cell sample image; a standard feature library creating module 12, configured to input the sample data set to a feature coding network, obtain a sample coding feature result, use the sample coding feature result as an abnormal cell image feature, and create a standard feature library representing the abnormal cell image feature according to an association relationship between the abnormal cell image feature and an abnormal description text; and the abnormal feature retrieval module 13 is configured to input the sample image to be retrieved into the feature coding network, obtain a feature result of the code to be retrieved, further obtain an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library, and complete retrieval of the abnormal cell image features.

In one embodiment, the standard feature library creation module 12 further comprises: the system comprises a sample set creating unit, a cell sample image processing unit and a cell sample image processing unit, wherein the sample set creating unit is used for sorting cell sample images corresponding to specific diseases, classifying and labeling the cell sample images according to preset disease classification standards and creating a sample data set; the network training unit is used for selecting a specified number of samples from the sample data set to train the feature coding network; and the characteristic acquisition unit is used for carrying out classification detection on all sample images in the sample data set according to the trained characteristic coding network to acquire corresponding abnormal cell image characteristics.

In an embodiment, the apparatus further comprises: the description text creating module is used for creating an abnormal description text of each feature according to a specific disease classification standard; the description text matching module is used for matching the description texts of the corresponding categories according to the labeling information of the cell sample images to obtain the abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories; and the association correction module is used for acquiring the sample image and the corresponding abnormal description text, outputting the sample image and the corresponding abnormal description text to a specific target object for verification, and judging whether the sample image is correctly associated with the abnormal description text.

In one embodiment, the feature encoding network comprises: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit; the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel; the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

In an embodiment, the channel attention unit is configured to obtain an evaluation score of each channel feature map as a channel attention, introduce the evaluation score into a corresponding channel output, and obtain an attention vector of a corresponding feature extraction channel; and weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-connection layer of the coding unit.

In one embodiment, the channel attention unit comprises a global average pooling layer, a first fully-connected layer, a first activation function, a second fully-connected layer and a second activation function which are connected in sequence; inputting the feature map obtained by the feature extraction channel into the global average pooling layer to obtain a scalar of the corresponding channel; and after the scalar sequentially passes through a first full connection layer, a first activation function, a second full connection layer and a second activation function, obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel, wherein the first activation function adopts a ReLU function, and the second activation function adopts a Sigmoid function.

In an embodiment, the fully-connected layer of the encoding unit is two layers, and includes a third fully-connected layer and a fourth fully-connected layer, and the third fully-connected layer outputs the acquired feature map to the fourth fully-connected layer and the hash layer, respectively; and the fourth full connection layer outputs the acquired feature map to the hash layer.

The above-mentioned abnormal cell image feature retrieval means may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 5. A computer device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor.

All or part of the modules in the abnormal cell image feature retrieval device can be realized by software, hardware and a combination thereof. The modules can be embedded in a memory of the terminal in a hardware form or independent from the memory of the terminal, and can also be stored in the memory of the terminal in a software form, so that the processor can call and execute the corresponding operations of the modules. The processor can be a Central Processing Unit (CPU), a microprocessor, a singlechip and the like.

Fig. 6 is a schematic diagram of an internal structure of the computer device in one embodiment. There is provided a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: constructing a feature coding network for acquiring binary coding features of an input image; acquiring a sample data set, wherein the sample data set comprises a cell sample image; inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and an abnormal description text; and inputting the sample image to be retrieved into the feature coding network to obtain a feature result of the code to be retrieved, and further obtaining an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library to complete the retrieval of the abnormal cell image features.

In an embodiment, when the processor executes the above method, the inputting the sample data set into a feature coding network, obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature includes: sorting cell sample images corresponding to a specific disease, classifying and labeling the cell sample images according to a preset disease classification standard, and creating a sample data set; selecting a specified number of samples from the sample data set to train the feature coding network; and carrying out classification detection on all sample images in the sample data set according to the trained feature coding network to obtain corresponding abnormal cell image features.

In an embodiment, before the above processor, when executing, inputting the sample data set to a feature coding network, obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature, the method further includes: creating an abnormal description text of each feature according to a specific disease classification standard; matching description texts of corresponding categories according to the labeling information of the cell sample images to obtain abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories; and acquiring the sample image and the corresponding abnormal description text, outputting the sample image and the corresponding abnormal description text to a specific target object for verification, and judging whether the sample image is correctly associated with the abnormal description text.

In an embodiment, when the processor executes, the feature encoding network implemented includes: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit; the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel; the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

In an embodiment, when the processor executes the above method, the performing feature extraction by using each feature extraction channel of the multi-scale attention unit includes: performing convolution on the input image through the convolution layers of the channels respectively to obtain a feature map output by each convolution layer; setting a channel attention unit, acquiring the evaluation score of each channel feature map as channel attention, introducing the evaluation score into the output of a corresponding channel, and acquiring an attention vector of a corresponding feature extraction channel; and weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-connection layer of the coding unit.

In an embodiment, when the processor executes, the implemented channel attention unit includes a global average pooling layer, a first full-connection layer, a first activation function, a second full-connection layer, and a second activation function, which are connected in sequence; inputting the feature map obtained by the feature extraction channel into the global average pooling layer to obtain a scalar of the corresponding channel; and after the scalar sequentially passes through a first full connection layer, a first activation function, a second full connection layer and a second activation function, obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel, wherein the first activation function adopts a ReLU function, and the second activation function adopts a Sigmoid function.

In an embodiment, when the processor executes the above-mentioned processing, the implemented full-connected layer of the coding unit is two layers, including a third full-connected layer and a fourth full-connected layer, and the third full-connected layer outputs the obtained feature map to the fourth full-connected layer and the hash layer, respectively; and the fourth full connection layer outputs the acquired feature map to the hash layer.

In one embodiment, the computer device may be used as a server, including but not limited to a stand-alone physical server or a server cluster formed by a plurality of physical servers, and may also be used as a terminal, including but not limited to a mobile phone, a tablet computer, a personal digital assistant or a smart device. As shown in fig. 6, the computer apparatus includes a processor, a nonvolatile storage medium, an internal memory, a display screen, and a network interface, which are connected by a system bus.

Wherein, the processor of the computer device is used for providing calculation and control capability and supporting the operation of the whole computer device. A non-volatile storage medium of the computer device stores an operating system and a computer program. The computer program can be executed by a processor to implement a method for retrieving abnormal cell image features provided in the above embodiments. The internal memory in the computer device provides a cached execution environment for the operating system and computer programs in the non-volatile storage medium. The display interface can display data through the display screen. The display screen may be a touch screen, such as a capacitive screen or an electronic screen, and the corresponding instruction may be generated by receiving a click operation applied to a control displayed on the touch screen.

Those skilled in the art will appreciate that the configuration of the computer device shown in fig. 6 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation of the computer device to which the present application applies, and that a particular computer device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, performs the steps of: constructing a feature coding network for acquiring binary coding features of an input image; acquiring a sample data set, wherein the sample data set comprises a cell sample image; inputting the sample data set into a feature coding network, obtaining a sample coding feature result, taking the sample coding feature result as an abnormal cell image feature, and establishing a standard feature library for representing the abnormal cell image feature according to the incidence relation between the abnormal cell image feature and an abnormal description text; and inputting the sample image to be retrieved into the feature coding network to obtain a feature result of the code to be retrieved, and further obtaining an abnormal description text matched with the feature result of the code to be retrieved through the standard feature library to complete the retrieval of the abnormal cell image features.

In one embodiment, the computer program, when executed by a processor, implements inputting the sample data set into a feature coding network, obtaining a sample coding feature result, and using the sample coding feature result as an abnormal cell image feature, including: sorting cell sample images corresponding to a specific disease, classifying and labeling the cell sample images according to a preset disease classification standard, and creating a sample data set; selecting a specified number of samples from the sample data set to train the feature coding network; and carrying out classification detection on all sample images in the sample data set according to the trained feature coding network to obtain corresponding abnormal cell image features.

In an embodiment, before the computer program, when executed by a processor, is implemented to input the sample data set into a feature coding network, obtain a sample coding feature result, and use the sample coding feature result as an abnormal cell image feature, the method further includes: creating an abnormal description text of each feature according to a specific disease classification standard; matching description texts of corresponding categories according to the labeling information of the cell sample images to obtain abnormal description texts of the corresponding cell sample images, wherein the labeling information comprises abnormal characteristic categories; and acquiring the sample image and the corresponding abnormal description text, outputting the sample image and the corresponding abnormal description text to a specific target object for verification, and judging whether the sample image is correctly associated with the abnormal description text.

In one embodiment, the computer program, when executed by a processor, implements the feature encoding network comprising: the system comprises a multi-scale feature extraction unit, a channel attention unit and a coding unit; the multi-scale attention unit comprises at least four mutually independent feature extraction channels, and the channel attention unit acts on each feature extraction channel to obtain the attention vector of each feature extraction channel; the coding unit comprises a full connection layer and a hash layer; and receiving the characteristic graph output by the attention increasing unit through each neuron of the full connection layer to obtain the abnormal characteristic of the input image, and inputting the global characteristic into the Hash layer to carry out binary coding to obtain the corresponding abnormal cell image characteristic.

In one embodiment, the computer program, when executed by a processor, implements feature extraction using feature extraction channels of a multi-scale attention unit, including: performing convolution on the input image through the convolution layers of the channels respectively to obtain a feature map output by each convolution layer; setting a channel attention unit, acquiring the evaluation score of each channel feature map as channel attention, introducing the evaluation score into the output of a corresponding channel, and acquiring an attention vector of a corresponding feature extraction channel; and weighting the attention of the channel into the feature map of the corresponding channel to obtain a corrected feature map, and inputting the corrected feature map into the full-connection layer of the coding unit.

In an embodiment, the computer program, when executed by a processor, implements the channel attention unit comprising a global average pooling layer, a first fully-connected layer, a first activation function, a second fully-connected layer, and a second activation function connected in series; inputting the feature map obtained by the feature extraction channel into the global average pooling layer to obtain a scalar of the corresponding channel; and after the scalar sequentially passes through a first full connection layer, a first activation function, a second full connection layer and a second activation function, obtaining the scalar between 0 and 1 of the corresponding channel as the attention weight of the channel, wherein the first activation function adopts a ReLU function, and the second activation function adopts a Sigmoid function.

In an embodiment, when the computer program is executed by a processor, the implemented full-connected layer of the coding unit is two layers, including a third full-connected layer and a fourth full-connected layer, and the third full-connected layer outputs the acquired feature map to the fourth full-connected layer and the hash layer respectively; and the fourth full connection layer outputs the acquired feature map to the hash layer.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.

In summary, the invention provides a method, a device, equipment and a medium for searching abnormal cell image features, wherein a standard typical cervical abnormal cell sample library is established strictly according to a TBS system, and experts perform final review screening and write corresponding cell feature descriptions, so that a convenient and practical consultation search platform is provided for cytopathologists; by adopting a network of a high-efficiency pyramid segmentation feature extraction module added with an attention mechanism, multi-scale spatial information and a cross-channel attention mechanism are integrated, so that pathological image features of cervical cancer cells are better fitted, detailed information of the cervical cancer cells is better extracted, and accuracy of feature retrieval is improved; meanwhile, binary hash code coding is adopted, and the retrieval efficiency is improved. For doctors with insufficient clinical diagnosis experience, similar typical examples are searched in a pathology database, the description and judgment of experts can be referred, data reference is provided for current image feature judgment, and comprehensive analysis can be made by combining historical pathology database diagnosis results. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention. .