阅读说明：本技术 用于生成电子喉镜医学检测报告的系统和方法 (System and method for generating electronic laryngoscope medical test reports ) 是由 潘晓英 陈皓 代栋 刘星星 闫庆元 孙雪华 于 2019-11-19 设计创作，主要内容包括：本发明涉及一种用于生成电子喉镜医学检测报告的系统和方法,该系统由数据处理模块、图像筛选模块、器官分割模块和病灶检测模块组成,所述数据处理模块与图像筛选模块相接,所述图像筛选模块分别接于器官分割模块和病灶检测模块；所述图像筛选模块、器官分割模块和病灶检测模块中分别设置有分类模型、分割模型和检测模型,器官分割模块和病灶检测模块分别接于器官定位模块。本发明提供的系统和方法能够在背景复杂的情况下识别出病灶并标记,生成诊断报告。并且可以针对不同器官的不同病变识别；预处理单元可以实时的从检查设备中读取检查视频并进行初步处理。(The invention relates to a system and a method for generating an electronic laryngoscope medical detection report, wherein the system consists of a data processing module, an image screening module, an organ segmentation module and a focus detection module, the data processing module is connected with the image screening module, and the image screening module is respectively connected with the organ segmentation module and the focus detection module; the image screening module, the organ segmentation module and the focus detection module are respectively provided with a classification model, a segmentation model and a detection model, and the organ segmentation module and the focus detection module are respectively connected with the organ positioning module. The system and the method provided by the invention can identify and mark the focus under the condition of complex background, and generate a diagnosis report. And can be identified for different lesions of different organs; the preprocessing unit can read the inspection video from the inspection device in real time and perform preliminary processing.)

1. A system for generating medical detection reports of an electronic laryngoscope is characterized by comprising a data processing module, an image screening module, an organ segmentation module and a focus detection module, wherein the data processing module is connected with the image screening module which is respectively connected with the organ segmentation module and the focus detection module; the image screening module, the organ segmentation module and the focus detection module are respectively provided with a classification model, a segmentation model and a detection model, and the organ segmentation module and the focus detection module are respectively connected with the organ positioning module.

2. The method of generating an electronic laryngoscope medical examination report according to claim 1, comprising the steps of:

the method comprises the following steps: establishing a medical detection report template and a disease description template;

step two: data set preparation: reading an examination video in the electronic endoscope of the hospital, transmitting the read data into the third step for preprocessing, and forming training data by the image data obtained after the preprocessing;

step three: image preprocessing:

s301: slicing the video frame by frame, intercepting an image every 3 frames, and handing over to S302 for processing;

s302: cutting the image, identifying a circular or rectangular frame for displaying the laryngoscope image by an image processing method, automatically cutting off useless areas outside the frame, and reserving the image part in the frame;

s303: in vitro and blur removal: training a classification model for clipping in-vitro and fuzzy segments in a video, dividing images after video frame splitting into effective images and ineffective images, wherein the ineffective images comprise in-vitro or fuzzy images, naming the effective images according to the naming principle of S304 after the ineffective images are removed, and storing the effective images;

s304: image naming, image tracing: uniformly naming the preprocessed image data in a mode of patient number _ time _ frame number. jpg;

step four: model training:

classification models: selecting images with clear organ parts and centered positions from the data generated in the step three, dividing the images into different label categories according to the categories of the organs, training an image classification model by adopting the image data after the categories are divided, wherein the model is used for dividing the detection images into different categories according to the organs and screening a plurality of images with the highest classification accuracy from each category;

and (3) segmenting the model: selecting an image with clear organ parts from the data generated in the step three, marking the organ edges in the image by using an irregular polygon, marking the organ types, taking the marked image data as a training sample to train an organ segmentation model, and using the model to identify and segment the organs in the detected image and output the organ types and contour coordinates;

and (3) detecting the model: selecting an image with a focus from the data generated in the step three, marking the position of the focus by using a rectangular frame, marking the name of the focus, training a focus detection model by using the marked image data as a training sample, wherein the model is used for identifying the focus in the detected image and outputting the focus category and the coordinate range to an organ positioning module;

step five: detection was performed using the pre-trained model in step four:

s501: data entry: the system reads a video stream from the electronic endoscope detection equipment in real time and sends the video stream to the data processing module for data processing;

s502: image screening: classifying the preprocessed images by using a classification model through the classification model in the image screening module, outputting a plurality of images with the highest accuracy of each target part according to the classification result of the model, and simultaneously sending the result to the organ segmentation module and the focus detection module;

s503: organ segmentation: reading the screened effective image data from the image screening module, detecting organ parts in the image by adopting a pre-trained segmentation model, segmenting the detected organ, and outputting the detected organ type and the coordinate range of the organ parts in the current image;

s504: detecting a focus: reading the screened effective image data from the image screening module, carrying out focus detection on the image by adopting a pre-trained detection model, and selecting one or more images with the highest target part accuracy rate as typical images to be output by combining the accuracy rate of a detection result and the accuracy rate of a segmentation effect in the image segmentation step; outputting the type and coordinate range of the focus to an organ positioning module when the focus is identified, and jumping to the seventh step when the focus is not identified to generate an abnormal case report;

step six: locating a focus organ:

detecting the organ name and coordinate range output to the organ positioning module by the organ segmentation module and the focus type and focus range returned by the focus detection module to generate focus description;

step seven: report generation: according to the detection result, standard descriptions corresponding to organs, pathological changes and abnormal conditions are matched in the symptom description template established in the step one, information of a current patient, detection equipment information, detection doctor information and the like are read from a database and inserted into the corresponding position of the detection report template generated in the step one, the typical image which can clearly reflect illness state information and is output in the step six is inserted into the corresponding position of the detection report template, and the template is stored as a pdf format file and output.

The technical field is as follows:

the present invention relates to the field of intelligent medical applications, and in particular to a system and method for generating electronic laryngoscope medical test reports.

Background art:

computer technology and related artificial intelligence technology are rapidly developing today, and are also widely used in the medical field. The electronic laryngoscope is the most direct and effective means for observing the mucous membrane of the visceral organ in the cavity, has the characteristics of light and handy lens body, thinness, flexibility and the like, flexible following performance and better insertion performance, enables the lens to enter the laryngeal cavity to be closer to a diseased part, can clearly see the fine change of a respiratory tract, and realizes faster diagnosis and treatment. Meanwhile, the electronic laryngoscope adopts a leading optical digital technology to provide high-definition picture quality, and does not need to adjust and automatically adjust light, which becomes an important condition for the application of an image processing technology on the electronic laryngoscope. At present, the electronic laryngoscope mainly plays the roles of diagnosis and minimally invasive diagnosis and treatment in clinic and becomes an important tool in the hands of otolaryngology-head and neck surgeons.

The existing electronic endoscope examination has some defects, firstly, the existing examination depends on the experience of doctors, the examining doctors determine the pathological changes by observing endoscope images, and the imbalance phenomenon of the medical level in China is serious; secondly, the endoscope examination needs attention to be very concentrated, the missed detection and the false detection of a doctor are easy to occur, the nasopharynx throat position is exquisite, and the cost of the missed detection and the false detection of a patient is high; finally, the need for the physician to manually add the corresponding condition and description to the report increases the time of the examination, resulting in inefficiencies. The existing automatic report generating system is mostly used for simple backgrounds of alimentary tracts and the like, the background of the nasopharynx part is complex, and the simple automatic report generating system can not meet the requirement of the complex background; existing medical image processing systems or methods identify lesions only to a single organ; the existing data processing cycle is long, and the requirement of a hospital on examination real-time performance cannot be well met.

The invention content is as follows:

the invention provides a system and a method for generating an electronic laryngoscope medical detection report, which aim to solve the problems that the prior art can only identify pathological changes of a single organ, has long data processing period and can not well meet the requirement of a hospital on examination real-time property.

In order to realize the purpose of the invention, the technical scheme provided by the invention is as follows:

a system for generating medical detection reports of an electronic laryngoscope comprises a data processing module, an image screening module, an organ segmentation module and a focus detection module, wherein the data processing module is connected with the image screening module; the image screening module, the organ segmentation module and the focus detection module are respectively provided with a classification model, a segmentation model and a detection model, and the organ segmentation module and the focus detection module are respectively connected with the organ positioning module.

The method for generating the electronic laryngoscope medical detection report by the system comprises the following steps:

the method comprises the following steps: establishing a medical detection report template and a disease description template;

step two: data set preparation: reading an examination video in the electronic endoscope of the hospital, transmitting the read data into the third step for preprocessing, and forming training data by the image data obtained after the preprocessing;

step three: image preprocessing:

s301: slicing the video frame by frame, intercepting an image every 3 frames, and handing over to S302 for processing;

s302: cutting the image, identifying a circular or rectangular frame for displaying the laryngoscope image by an image processing method, automatically cutting off useless areas outside the frame, and reserving the image part in the frame;

s303: in vitro and blur removal: training a classification model for clipping in-vitro and fuzzy segments in a video, dividing images after video frame splitting into effective images and ineffective images, wherein the ineffective images comprise in-vitro or fuzzy images, naming the effective images according to the naming principle of S304 after the ineffective images are removed, and storing the effective images;

s304: image naming, image tracing: uniformly naming the preprocessed image data in a mode of patient number _ time _ frame number. jpg;

step four: model training:

classification models: selecting images with clear organ parts and centered positions from the data generated in the step three, dividing the images into different label categories according to the categories of the organs, training an image classification model by adopting the image data after the categories are divided, wherein the model is used for dividing the detection images into different categories according to the organs and screening a plurality of images with the highest classification accuracy from each category;

and (3) segmenting the model: selecting an image with clear organ parts from the data generated in the step three, marking the organ edges in the image by using an irregular polygon, marking the organ types, taking the marked image data as a training sample to train an organ segmentation model, and using the model to identify and segment the organs in the detected image and output the organ types and contour coordinates;

and (3) detecting the model: selecting an image with a focus from the data generated in the step three, marking the position of the focus by using a rectangular frame, marking the name of the focus, training a focus detection model by using the marked image data as a training sample, wherein the model is used for identifying the focus in the detected image and outputting the focus category and the coordinate range to an organ positioning module;

step five: detection was performed using the pre-trained model in step four:

s501: data entry: the system reads a video stream from the electronic endoscope detection equipment in real time and sends the video stream to the data processing module for data processing;

s502: image screening: classifying the preprocessed images by using a classification model through the classification model in the image screening module, outputting a plurality of images with the highest accuracy of each target part according to the classification result of the model, and simultaneously sending the result to the organ segmentation module and the focus detection module;

s503: organ segmentation: reading the screened effective image data from the image screening module, detecting organ parts in the image by adopting a pre-trained segmentation model, segmenting the detected organ, and outputting the detected organ type and the coordinate range of the organ parts in the current image;

s504: detecting a focus: reading the screened effective image data from the image screening module, carrying out focus detection on the image by adopting a pre-trained detection model, and selecting one or more images with the highest target part accuracy rate as typical images to be output by combining the accuracy rate of a detection result and the accuracy rate of a segmentation effect in the image segmentation step; outputting the type and coordinate range of the focus to an organ positioning module when the focus is identified, and jumping to the seventh step when the focus is not identified to generate an abnormal case report;

step six: locating a focus organ:

detecting the organ name and coordinate range output to the organ positioning module by the organ segmentation module and the focus type and focus range returned by the focus detection module to generate focus description;

step seven: report generation: according to the detection result, standard descriptions corresponding to organs, pathological changes and abnormal conditions are matched in the symptom description template established in the step one, information of a current patient, detection equipment information, detection doctor information and the like are read from a database and inserted into the corresponding position of the detection report template generated in the step one, the typical image which can clearly reflect illness state information and is output in the step six is inserted into the corresponding position of the detection report template, and the template is stored as a pdf format file and output.

Compared with the prior art, the invention has the remarkable improvements that:

the system and the method provided by the invention can identify and mark the focus under the condition of complex background, and some methods for detecting and identifying the endoscope lesion in the digestive tract exist, but the structure change of the digestive tract is small, and the background of the focus is simple. The nasopharynx and laryngeal part has large structural change, including nasal cavity, nasopharynx, epiglottis and larynx, and each part has large structural difference, various pathological changes and complex focus background. The invention can detect the focus under the complex environment and generate a diagnosis report. And can be identified for different lesions of different organs; the preprocessing unit can read the inspection video from the inspection device in real time and perform preliminary processing.

Description of the drawings:

FIG. 1 is a block diagram of the system of the present invention.

The specific implementation mode is as follows:

the present invention will be described in detail below with reference to the drawings and examples.

Referring to fig. 1, the system for generating an electronic laryngoscope medical examination report provided by the invention comprises a data processing module, a picture screening module, an organ segmentation module and a focus detection module, wherein the data processing module is connected with the picture screening module, and the picture screening module is respectively connected with the organ segmentation module and the focus detection module; and the image screening module, the organ segmentation module and the focus detection module are respectively provided with a classification model, a segmentation model and a detection model.

The method for generating the electronic laryngoscope medical detection report by the system comprises the following steps:

the method comprises the following steps: and establishing a medical detection report template and a disease description template. The medical detection report template is designed according to the writing requirements of the case report of the hospital, placeholders are arranged on corresponding information input positions in the template, and corresponding information is inserted in a report generation stage; disorder description templates for example: 'polyp above left vocal cords', etc.

Step two: data entry: reading an examination video in an electronic endoscope of a hospital in real time, and then transmitting the read video stream to a data processing module for processing;

step three: the image preprocessing process is divided into the following four steps

S301: slicing the video stream frame by frame, storing a picture every 3 frames, and handing over to S302 for processing;

s302: cutting picture data, cutting the picture data, identifying a circular or rectangular frame for displaying a laryngoscope image through an image processing method, automatically cutting off useless areas outside the frame, and only keeping an image part in the frame;

s303: in vitro and blur removal: training a classification model for clipping in-vitro and fuzzy segments in a video, dividing a frame-splitting picture of the video into effective and ineffective pictures according to whether the frame-splitting picture is effective or not, wherein the ineffective picture is in-vitro or fuzzy picture, removing the ineffective picture, naming the effective pictures according to the naming principle of S304, and storing the effective pictures in a database;

s304: image naming, image tracing: the method of patient name _ time _ frame number jpg is uniformly adopted for naming the preprocessed picture data, the picture on the report can be associated through the step, errors are avoided to a certain extent, and the source video information of the current image can be obtained.

Step four: model training:

classification models: selecting images with clear organ parts and centered positions from the data generated in the step three, dividing the images into different label categories according to the categories of the organs, training an image classification model by adopting the image data after the categories are divided, wherein the model is used for dividing the detection images into different categories according to the organs and screening a plurality of images with the highest classification accuracy from each category;

and (3) segmenting the model: selecting an image with clear organ parts from the data generated in the step three, marking the organ edges in the image by using an irregular polygon, marking the organ types, taking the marked image data as a training sample to train an organ segmentation model, and using the model to identify and segment the organs in the detected image and output the organ types and contour coordinates;

and (3) detecting the model: selecting an image with a focus from the data generated in the third step, marking the position of the focus by using a rectangular frame, marking the name of the focus, training a focus detection model by using the marked image data as a training sample, wherein the model is used for identifying the focus in the detected image and outputting the focus category and the coordinate range to an organ positioning module.

Step five: the detection was performed using the model in step four:

s501: data entry: the system reads the video stream from the electronic endoscope detection equipment in real time and sends the video stream to the data processing module for data processing.

S502: image screening: classifying the preprocessed pictures by using a classification model through the classification model in the picture screening module, outputting a plurality of pictures with the highest accuracy of each target part according to the classification result of the model, and simultaneously returning the result to the organ segmentation module and the focus detection module;

s503: organ segmentation: reading effective image data screened from the picture screening from a picture screening module, detecting a target part in an image by adopting a pre-trained segmentation model, segmenting the target part by an organ segmentation module, and outputting the detected coordinate range of each organ and the organ in the current image;

s504: detecting a focus: reading effective image data screened in the image screening step from the image screening module, carrying out target detection on the image by using a pre-trained detection model, and selecting one or more images with the highest target part accuracy rate as typical images to be output by combining the accuracy rate of a detection result and the accuracy rate of a segmentation effect in the image segmentation step; and when the focus is identified, outputting the focus type and the coordinate range to an organ positioning module, and when the focus is not detected, skipping to the seventh step to generate an abnormal case report.

Step six: locating a focus organ:

and if the focus is detected at the S503 stage in the fifth step, determining the organ and the position of the focus position according to the organ name and the coordinate range output to the organ positioning module by the S502 organ segmentation module and the focus type and the focus range returned by the S503 focus detection module, and generating a focus description.