Method and system for image analysis of medical images
阅读说明：本技术 用于医学图像的图像分析的方法和系统 (Method and system for image analysis of medical images ) 是由 B·查克拉巴蒂 P·拉古塔姆文卡特 R·巴特 于 2018-06-21 设计创作，主要内容包括：本发明涉及用于医学图像的图像分析的方法和系统。所述方法包括接收医学图像、所述医学图像的第一注释和第一ROI信息,并且接收所述医学图像的第二注释、第二ROI信息和查看数据,已经基于所述查看数据做出所述第二注释。随后,所述方法包括通过比较所述第二ROI与所述第一ROI信息并且通过检查所述查看数据来检查所述第二注释的质量。(The invention relates to a method and a system for image analysis of medical images. The method includes receiving a medical image, a first annotation and first ROI information of the medical image, and receiving a second annotation, second ROI information and viewing data of the medical image, the second annotation having been made based on the viewing data. Subsequently, the method includes checking the quality of the second annotation by comparing the second ROI with the first ROI information and by checking the viewing data.)
1. A method for image analysis of a medical image (100), the method comprising:
-receiving a medical image (100), a first annotation (11) of the medical image and first region of interest, ROI, information (12) indicative of a first ROI within the medical image (100),
-receiving a second annotation (21) of the medical image (100), second ROI information (22) and viewing data (23), the second ROI information being indicative of a second ROI within the medical image (100), the viewing data comprising display data and/or a viewing time (234), the second ROI having been displayed on a display together with the display data and/or viewing time and the second annotation (21) having been made based on the display data and/or viewing time, and
-checking the quality of the second annotation (21) by comparing the second ROI information (22) with the first ROI information (12) and by checking the viewing data (23).
2. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,
wherein checking the viewing data (23) comprises checking whether the viewing data (23) meets a viewing data requirement (40).
3. The method according to any one of the preceding claims,
wherein the viewing data (23) further comprises a zoom level (231) and/or a pan level (232), and/or
Wherein the display data comprises a resolution (233) and/or a display size (236) and/or a number of pixels (235), in particular in the display and/or the second ROI.
4. The method according to claim 2 and 3,
wherein the viewing data requirement 40 comprises a viewing time threshold (44) and/or a zoom level threshold (41) and/or a pan level threshold (42) and/or a resolution threshold (43) and/or a number of pixels threshold (45) and/or a display size threshold (46).
5. The method according to any one of the preceding claims,
wherein the ROI information (12, 22) comprises position information and/or size information indicating a position and/or size of the ROI.
6. The method according to any one of the preceding claims,
wherein the first annotation (11) is provided by one or more users and/or generated by automated image analysis.
7. The method according to any one of the preceding claims,
also comprises
-issuing the result of the quality check (4), and/or
-adjusting the image (100) according to the quality check (4).
8. The method according to any one of the preceding claims,
also comprises
-receiving further annotations (31) of the medical image, further ROI information (32) and further viewing data (33), the further ROI information being indicative of one or more further ROIs within the medical image (100), the further viewing data comprising display data and/or viewing times, the further ROIs having been displayed on a display together with the display data and/or viewing times comprised by the further viewing data, and the further annotations (31) having been made based on the display data and/or viewing times comprised by the further viewing data, and
-checking the quality of the further annotation (32) by comparing the further ROI information (32) with the first and/or second ROI information (12, 22) and by checking the further viewing data (33).
9. The method according to any one of the preceding claims,
wherein the first annotation (11) and the first ROI information (12) are based on that the medical image (100) has been observed on a high resolution display, and wherein the second ROI has been displayed on a low resolution display.
10. A system (5) for image analysis of a medical image (100), the system comprising:
a first input unit (6) configured to receive a medical image (100), a first annotation (11) of the medical image (100) and first region of interest, ROI, information (12) indicative of a first ROI within the medical image (100),
-a second input unit (7) configured to receive a second annotation (22) of the medical image (100), second ROI information (22) and viewing data (23), the second ROI information being indicative of a second ROI within the medical image (100), the viewing data comprising display data and/or a viewing time (234), the second ROI having been displayed on a display together with the display data and/or viewing time and the second annotation (22) having been made based on the display data and/or viewing time, and
-a quality checking unit (8) configured to check the quality of the second annotation (21) by comparing the second ROI information (22) with the first ROI information (12) and checking the viewing data (23).
11. The system of claim 10, wherein the first and second light sources are arranged in a single package,
also comprises
-an output unit (9) configured to issue a result of the quality check (4), and/or
-an adjustment unit (10) configured to adjust the image (100) in accordance with the quality check (4).
12. The system of any one of claims 10 to 11,
wherein the second input unit (7) is further configured to: receiving further annotations (31) of the medical image (100), further ROI information (32) and further viewing data (23), the further ROI information being indicative of one or more further ROIs within the medical image (100), the further viewing data comprising display data and/or viewing times, the further ROIs having been displayed on a display together with the display data and/or viewing times comprised by the further viewing data, and the further annotations (31) having been made based on the display data and/or viewing times comprised by the further viewing data; and checking the quality of the further annotation (32) by comparing the further ROI information (32) with the first and/or second ROI information (12, 22) and checking the further viewing data (33).
13. A computer program comprising program code means for causing a computer to carry out the steps of the method as claimed in any one of claims 1 to 9.
14. A method for recording image-related information related to a medical image (100), the method comprising:
-receiving the medical image (100),
-displaying the medical image (100) on a display,
-recording annotations (21) made for the medical image (100),
-detecting and recording region of interest, ROI, information (22) indicative of a ROI within the medical image (100), displaying the medical image (100) together with the region of interest, ROI information, and making the annotation (21) based on the region of interest, and
-detecting and recording viewing data (23), the viewing data (23) comprising a viewing time, displaying the ROI with the viewing data, and making the annotation (21) based on the viewing data.
15. A device (70) for recording image-related information related to a medical image (100), the device comprising:
a receiver (75) configured to receive the medical image (100),
a display (76) configured to display the medical image (100), an
-a detection and recording unit (77) configured to:
-recording annotations (21) made for the medical image (100),
-detecting and recording region of interest, ROI, information (22) indicative of a ROI within the medical image (100), displaying the medical image (100) together with the region of interest, ROI information, and making the annotation (21) based on the region of interest, and
-detecting and recording viewing data (23), the viewing data comprising a viewing time (234), displaying the ROI with the viewing data, and making the annotation (21) based on the viewing data.
Technical Field
The present invention relates to a method and a system for image analysis of a medical image, a computer program and a method and a device for providing an annotation of a medical image.
Background
In modern medicine, medical imaging has undergone significant improvements. It is a non-invasive technique used to collect a visual representation of the interior of the body for clinical analysis and medical intervention. Medical imaging is also used to collect visual representations of the function of organs and tissues. It seeks to reveal internal structures that are occluded by skin and bone, and to diagnose and treat disease.
Medical imaging relates to both conventional images and to images that involve movement (i.e. video), to two-dimensional images, and also to three-dimensional images. Medical imaging techniques include, for example, X-ray imaging (particularly computed tomography imaging), ultrasound imaging, and magnetic resonance imaging.
In a clinical setting, medical imaging generally corresponds to radiology, and the physician responsible for acquiring and interpreting the images is a radiologist.
Reliable imaging is of the highest importance for the radiologist's decision making and unnecessary procedures can be reduced. In general, surgical intervention may be avoided, for example, if a diagnostic imaging service, such as X-ray imaging, is available.
However, appropriate decisions are premised on high quality medical images. Although the images themselves may be of high quality, there may still be problems with the inability of the radiologist or other medical professional to access the adequate display. This situation may occur, for example, in a remote radiology or proxy service setting, where multiple readers attempt to interpret a particular medical image independently of each other using their respective displays on a mobile device or tablet, etc. The same applies to a busy radiologist on the move who consults his/her smartphone to get an image in order to provide a second opinion about the medical diagnosis.
In these cases, the due care of the medical diagnosis may no longer be ensured. Thus, medical negligence and, in the worst case, erroneous treatment may be caused. Furthermore, urgent medical treatments may have to be delayed.
Disclosure of Invention
It is an object of the present invention to provide a method and a system for image analysis of medical images which ensure a strict evaluation of the corresponding medical diagnosis.
It is a further object of the invention to provide a method and apparatus for providing annotation of medical images.
In a first aspect of the present invention a method for image analysis of a medical image is presented, the method comprising:
-receiving a medical image, a first annotation of the medical image and first region of interest, ROI, information, the first region of interest, ROI, information being indicative of a first ROI within the medical image,
-receiving a second annotation of the medical image, second ROI information and viewing data, the second ROI information being indicative of a second ROI within the medical image, the viewing data comprising display data and/or a viewing time, the second ROI having been displayed on a display together with the display data and/or viewing time and the second annotation having been made based on the display data and/or viewing time, and
-checking the quality of the second annotation by comparing the second ROI information with the first ROI information and by checking the viewing data.
In a further aspect of the invention, a system for image analysis of medical images is presented, the system comprising:
a first input unit configured to receive a medical image, a first annotation of the medical image and first region of interest, ROI, information, the first region of interest, ROI, information being indicative of a first ROI within the medical image,
-a second input unit configured to receive a second annotation of the medical image, second ROI information and viewing data, the second ROI information being indicative of a second ROI within the medical image, the viewing data comprising display data and/or a viewing time, the second ROI having been displayed on a display together with the display data and/or viewing time and the second annotation having been made based on the display data and/or viewing time, and
-a quality checking unit configured to check the quality of the second annotation by comparing the second ROI information with the first ROI information, and to check the viewing data.
In a further aspect of the invention, a computer program is provided comprising program code means for performing the steps of the method for image analysis when said computer program is executed on a computer, and a non-transitory computer-readable recording medium having stored therein a computer program product, which when executed by a processor causes the method disclosed herein to be performed.
Preferred embodiments of the invention are defined in the dependent claims. It shall be understood that the claimed system for image analysis and computer program and medium have similar and/or identical preferred embodiments as the claimed method, in particular as defined in the dependent claims and as disclosed herein. It is further to be understood that the claimed apparatus for providing annotation of medical images has similar and/or identical preferred embodiments as the claimed method.
The invention is based on the idea of ensuring a strict evaluation of a medical diagnosis based on an analysis of a medical image.
During the analysis of medical images, the reader does not always have access to an adequate display. In this case, medical professionals are often forced to base their medical diagnosis only on poorly displayed images, for example only in low resolution and/or on small screens.
This problem arises especially in remote radiology or agency service settings and when a second opinion about diagnosis is required (i.e. where clinical diagnosis or treatment has been difficult). Thus, the risk of erroneous diagnosis or even erroneous treatment and treatment delays is greatly increased.
The method for image analysis according to the present invention can solve these problems. In particular, the method ensures that the diagnosis of the radiologist based on the analysis of the medical image on the display has been carefully and correctly evaluated. For example in case a lesion has been detected by a previous reader, it is necessary to bring the correct image region ROI into the view of the display device of the current reader. Only then can the reader's comments be properly understood. Furthermore, for satisfactory examination, the medical images may need to be displayed according to viewing data transmitted from a previous reader. Otherwise, the possibility of detecting a lesion or other medical abnormality is suboptimal at best.
By introducing a method for image analysis of medical images, wherein the quality of the diagnosis of the second reader is checked, false diagnoses can be minimized and false treatments or treatment delays can be avoided.
In one embodiment of the method for image analysis, checking the viewing data comprises checking whether the viewing data meets a viewing data requirement. In order to check the quality of the reader's diagnosis, it is necessary to check not only whether he/she has selected the ROI as found by the previous or parallel reader, but also whether his/her viewing data meets the viewing data requirements. The requirements are typically specified by a prior or concurrent reader. However, the viewing data requirements may also be automatically determined by a computer program that analyzes the images.
In a further embodiment of the method for image analysis, the viewing data may further comprise a zoom level and/or a pan level. In a further embodiment, the display data may further comprise a resolution and/or a display size and/or a number of pixels in the screen and/or in the second ROI. That is, it is accurately monitored how the radiologist displays the medical image on his/her display device. In addition, the kind of display used is analyzed. For example, it may be recorded that for the first 5 seconds of display, only the top half of the image is viewed, and the display used has a 13 inch diagonal and 1280x800 resolution and an ROI covering 1000 pixels.
In an embodiment of the method for image analysis, the viewing data requirements comprise a viewing time threshold and/or a zoom level threshold and/or a pan level threshold and/or a resolution threshold and/or a number of pixels threshold and/or a display size threshold. It is unlikely that a radiologist viewing an image only superficially confirms a reliable diagnosis. Therefore, it must be ensured that the radiologist observes the medical image for a sufficient amount of time. The temporal threshold for complex lesions may be higher than for simple lesions. The location of small anomalies, such as small pieces of cancer tissue, may be detectable and evaluable only at certain zoom levels. To avoid ignoring such regions, it is important to comply with a certain scaling threshold. For two-dimensional as well as three-dimensional images, minimal translation may be required. Otherwise, the ROI may not be displayed properly. For example, cardiac dysfunction cannot be diagnosed from the perspective of the heart covered by ribs. Likewise, retinal diseases can only be found from a certain perspective. Further, there may be a display in which the image cannot be observed at all so that a lesion or disease can be appropriately diagnosed. All scaling, for example, does not help if the resulting resolution is too low to observe the ROI with sufficient quality. The same applies if the number of pixels displayed is too low or the display size is too small.
In a further embodiment, the ROI information of the method for image analysis comprises position information and/or size information indicating a position and/or size of an ROI. There are various forms of ROI information. The ROI information may be given directly, e.g. by a marker in the medical image, e.g. a red circle around what is considered a fracture. The ROI information may also be given in the form of coordinates. The ROI information includes important information about the injury or disease of the patient. First, the location and size of the area from which the patient's pain originates provides the physician with indispensable information.
In another embodiment of the method for image analysis, the first annotation is provided by one or more users and/or generated by automated image analysis. They may be provided in advance or in parallel with a second annotation by another reader.
In a further embodiment, the method for image analysis of medical images further comprises issuing results of a quality check and/or adapting the image according to the quality check. Whether the first and second ROIs match and see whether the data meets certain requirements is critical for subsequent treatment of the patient. To ensure that the inspection is completely passed, the results of the quality inspection may be sent to the reader. For example, the results may be displayed on a display or sent via an audio message. In case of a defect, the reader may adjust the image or even switch to another display. However, the required adjustment can also be supported by automation or even be performed fully automatically.
In a further embodiment, the method for image analysis of a medical image further comprises receiving further annotations of the medical image, further ROI information and further viewing data, the further ROI information being indicative of one or more further ROIs within the medical image, the further viewing data comprising display data and/or viewing time, the further ROIs having been displayed on a display together with the display data and/or viewing time and the further annotations having been made based on the display data and/or viewing time, and checking the quality of the further annotations by comparing the further ROI information with the first and/or second ROI information and by checking the further viewing data. In evaluating medical images, it may be necessary to involve a larger group of medical professionals that may be located worldwide. Each doctor will then determine the corresponding ROI and make the corresponding annotation. In order to then decide which diagnosis is correct, all ROIs need to be compared and the corresponding viewing data needs to be checked accordingly. For example, the positions of all image portions being displayed will be recorded and compared to each other. Further, the time corresponding to each viewport view may be recorded and matched against a predetermined time threshold.
In an embodiment of the image analysis method, the first annotation and the first ROI information are based on the medical image having been viewed on a high resolution display, and wherein the second ROI has been displayed on a low resolution display. I.e. the second annotation is based on the medical image displayed at low resolution. Since the treatment decision may be based on these second annotations only, a quality check of the second annotations is particularly important.
In an embodiment of the system for image analysis of medical images, the system further comprises: an output unit configured to issue a result of a quality check, and/or an adjustment unit configured to adjust the image according to the quality check. The output unit may be included in the display so that the result of the quality check may be displayed directly on the image. Likewise, the output unit may be configured to emit the result via an audio signal or message. The adjusting unit may be configured to adjust a portion of the observed image. In addition to scrolling or rotating, the adjustment unit may be configured to change the zoom level. The adjustment unit may also have a panning function for displaying a three-dimensional image.
In a further embodiment of the system, the second input unit is further configured to receive further annotations of the medical image, further ROI information and further viewing data, the further ROI information being indicative of one or more further ROIs within the medical image, the further viewing data comprising display data and/or viewing time, the further ROIs having been displayed on a display together with the display data and/or viewing time and the further annotations having been made based on the display data and/or viewing time, and to check the quality of the further annotations by comparing the further ROI information with the first and/or second ROI information and to check the further viewing data. The system may be a teleradiology system or a proxy service system in which three or more radiologists or other physicians evaluate medical images using their respective display devices.
In a further aspect, a method for recording image-related information related to a medical image is presented, the method comprising:
-receiving the medical image,
-displaying the medical image on a display,
-recording annotations made for the medical image,
-detecting and recording region of interest, ROI, information, the region of interest, ROI information being indicative of a ROI within the medical image, displaying the medical image together with the region of interest, ROI information, and making the annotation based on the region of interest, and
-detecting and recording viewing data, said viewing data comprising a viewing time, displaying said ROI with said viewing data, and making said annotation based on said viewing data.
In a further aspect, a device for recording image-related information related to a medical image is presented, the device comprising:
a receiver configured to receive the medical image,
a display configured to display the medical image, an
-a detection and recording unit configured to:
-recording annotations made for the medical image,
-detecting and recording region of interest, ROI, information, the region of interest, ROI information being indicative of a ROI within the medical image, displaying the medical image together with the region of interest, ROI information, and making the annotation based on the region of interest, and
-detecting and recording viewing data, said viewing data comprising a viewing time, displaying said ROI with said viewing data, and making said annotation based on said viewing data.
By means of the method and the device for recording image-related information, the viewing pattern of a user viewing medical images can be monitored. In addition to the ROI information, display data and viewing time data zoom level and/or pan level data may also be recorded. The annotations made may then be evaluated in view of the monitored observation pattern. The device may in particular be used as a second input unit of a system for image analysis of medical images.
Drawings
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiment(s) described hereinafter. In the following drawings:
figure 1 shows a flow chart of a first embodiment of a method for image analysis of medical images according to the invention,
figure 2 shows a flow chart of a second embodiment of a method for image analysis of medical images according to the invention,
figure 3 shows a schematic view of a first embodiment of a system for image analysis of medical images according to the invention,
figure 4 shows a schematic view of a second embodiment of a system for image analysis of medical images according to the invention,
figures 5A, 5B and 5C show the medical image as viewed on different displays at different zoom levels,
FIGS. 6A, 6B, and 6C show medical images with zoom guides for proper display, and
fig. 7 shows a schematic view of an embodiment of a device for recording image-related information related to a medical image.
Detailed Description
Fig. 1 shows a flow chart of a first embodiment of a method for image analysis of medical images according to the present invention. Assuming a
Fig. 2 shows a flow chart of a second embodiment of a method for image analysis of medical images according to the invention. Assuming a two-dimensional
Fig. 3 shows a schematic view of a first embodiment of a
Fig. 4 shows a schematic view of a second embodiment of a
Fig. 5A, 5B and 5C show medical images as viewed at different zoom levels on different displays. In particular, the figure shows a human breast including clinical abnormalities. Although fig. 5A shows the image as viewed on a first device having a screen diagonal of about 25 inches, the mobile device used to display fig. 5B and 5C has a display diagonal of about 8 inches. That is, anomalies as observed in FIG. 5A can be easily identified, while smaller windows as shown in FIG. 5B are too small to identify any anomalies. Thus, a radiologist receiving an image as depicted in fig. 5B cannot get a qualified opinion. However, if the zoom level is adjusted accordingly as shown in fig. 5C, the small display size is sufficient. That is, if the radiologist views the chest image on an 8-inch display, he/she can only access the anomaly if the zoom level is appropriate. To ensure that the radiologist uses a sufficient zoom level, the zoom level used may be recorded by the mobile device and sent to the quality check unit.
Fig. 6A, 6B and 6C show medical images with zoom guides for proper display. In particular, a human breast with lymphadenopathy is shown. Fig. 6A shows a medical image in a form unsuitable for diagnosis. However, if ROI information is provided along with the image according to the present invention, an appropriate diagnosis can be made. Fig. 6B depicts how the ROI information may be transmitted, i.e. by marking around anomalies in the image. The flag indicates guidance for zooming. Using ROI labeling to find the appropriate zoom level, lymphadenopathy can be displayed in an optimal manner. The corresponding image is depicted in fig. 6C.
Fig. 7 shows a schematic view of an embodiment of a
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
A computer program may be stored/distributed on a suitable non-transitory medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
Any reference signs in the claims shall not be construed as limiting the scope.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：用于超声系统独立衰减系数估计的方法