阅读说明：本技术 搜索医学参考图像 (Searching medical reference images ) 是由 斯文·科勒 克里斯蒂安·蒂特延 热拉尔多·埃尔莫斯罗·巴拉德斯 廖舒 费利克斯·里特尔 扬 于 2019-08-09 设计创作，主要内容包括：本发明涉及一种用于识别至少一个医学参考图像(REFI)的方法。该方法包括以下步骤：基于描绘第一患者的身体部位的当前检查图像(CURI)提供医学表示图像(REPI)(S1),限定表示图像中的感兴趣区域(ROI)(S2),至少针对表示图像中限定的感兴趣区域生成特征签名(FESI)(S4),基于所生成的特征签名将伪像图像与存储在医学图像数据库(8)中的至少一个第二患者的多个医学图像(MEDI)进行比较(S5),以及将数据库中的至少一个医学图像识别为医学参考图像(REFI)(S6),医学参考图像提供高于预定阈值的相对于表示图像的相似度。使用训练的机器学习算法(40)执行生成步骤。(The invention relates to a method for identifying at least one medical reference image (REFI). The method comprises the following steps: providing a medical representation image (REPI) based on a current examination image (CURI) depicting a body part of a first patient (S1), defining a region of interest (ROI) in the representation image (S2), generating a feature signature (FESI) at least for the region of interest defined in the representation image (S4), comparing the artifact image with a plurality of medical images (MEDI) of at least one second patient stored in a medical image database (8) based on the generated feature signature (S5), and identifying the at least one medical image in the database as a medical reference image (REFI) (S6), the medical reference image providing a similarity with respect to the representation image above a predetermined threshold. The generating step is performed using a trained machine learning algorithm (40).)

1. A method for identifying at least one medical reference image (REFI), comprising the steps of:

-providing a medical representation image (REPI) based on a current examination image (CURI) depicting a body part of the first patient (S1),

-defining a region of interest (ROI) in the representation image (S2),

-generating a feature signature (FESI) at least for the region of interest defined in the representation image (S4),

-comparing (S5) the representative image with a plurality of medical images (MEDI) of at least one second patient stored in a medical image database (8) based on the generated feature signature, and

-identifying at least one medical image in the database as the medical reference image (REFI) (S6), the medical reference image providing a similarity above a predetermined threshold with respect to the representation image,

wherein the step of generating is performed using a trained machine learning algorithm (40).

2. The method of claim 1, wherein providing the representation image comprises taking a digital photograph of the current inspection image.

3. The method of any of claims 1 or 2, wherein defining the region of interest is based on identifying an anatomical feature in the representation image, wherein the anatomical feature is indicative of a pathological condition of the patient.

4. The method according to any one of claims 1 to 3, wherein the definition of the region of interest is performed manually by a user.

5. The method according to any of the preceding claims, further comprising the step of:

-correcting at least one artifact in the representation image before generating the feature signature (S3).

6. The method of claim 5, wherein correcting the artifacts in the representative image comprises correcting at least one of: image element grid artifacts, dust artifacts, LCD refresh artifacts, illumination artifacts, and artifacts due to limited gray level dynamic range.

7. The method of claim 5 or 6, wherein correcting the artifact in the representation image is performed by a trained machine learning algorithm (40).

8. The method according to any of the preceding claims, wherein the steps of comparing and identifying are performed taking into account residual artifacts remaining after correcting the representative image.

9. The method of any preceding claim, further comprising:

-acquiring imaging parameters of the current examination image and each of the plurality of medical images, the imaging parameters being indicative of an imaging modality used for image acquisition, an

-identifying the reference image based on the identified imaging parameters.

10. A system (1) for identifying at least one medical reference image (REFI), comprising:

-an interface unit adapted to

Providing (S1) a representation image (REPI) on the basis of a current examination image (CURI) representing a body part of the first patient, an

-a calculation unit (7) adapted to

Defining (S2) a region of interest (ROI) in the representation image,

generating (S4) a feature signature (FESI) at least for the region of interest in the representation image,

comparing the representation image with a plurality of medical images stored in a medical image database (8) based on the generated feature signature (S5),

identifying (S6) at least one medical image in the database as the medical reference image (REFI), the medical reference image showing a similarity above a predetermined threshold with respect to the representation image,

wherein the computing unit (7) is adapted to run a trained machine learning algorithm (40) for performing the step of generating (S4).

11. The system as defined in claim 9, wherein the computing unit (7) is further adapted to correct (S3) at least one artifact in the representation image.

12. The system according to claim 10 or 11, adapted to implement the method for identifying at least one medical reference image according to any one of claims 1 to 9.

13. A computer program product comprising program elements which, when loaded into a memory of a computing unit (7) of a system (1), direct the computing unit (7) for identifying at least one medical reference image (REFI) for performing the steps of the method according to any one of claims 1 to 9.

14. A computer-readable medium having stored thereon a program element, which, when being executed by a computing unit (7) of a system (1), is readable and executable by the computing unit (7) for identifying at least one medical reference image (REFI) for performing the steps of the method according to any one of claims 1 to 9.

Technical Field

The present disclosure relates to the field of medical image processing, in particular to a method and system for identifying at least one medical reference image (REFI).

Background

In daily routine, radiologists are often faced with medical images depicting occasional special image patterns that are difficult to classify at first glance. However, these images need to be recorded in a corresponding radiology report. Thus, casual findings are described by their imaging features preferably using free text form without deduction of specific diagnoses. The radiologist may manually screen the clinical textbook, imaging atlas, or obtain a second comparable expert opinion, if possible, to classify the incidental finding. Appropriate differential diagnosis can then be applied. The procedure is independent of the technical standards present in radiology and therefore independent of film-based light box reading, reading in digital PACS viewers or making use of advanced visualization applications/sites such as

Is read. The volume of clinical studies, literature and/or clinical guidelines for differential diagnosis is enormous and therefore not likely to be known to individual radiologists. In addition, radiologists often work under strict time constraints.

Typically, image search portals are available and well known, such as Google image search. A single image may be uploaded and compared to images stored publicly in the internet. While these portals are generally not suitable for discovering diseases based on a given image modality of the medical image in question (e.g. due to the fact that known software solutions do not compatibly or compatibly provide the possibility to acquire a region of interest (ROI) to send it to the portal site), such an approach also requires the presence and close integration into the radiology reading hardware and software environment. Thus, only selected users with high-end equipment can benefit from the image search portal for radiology purposes.

In addition to this, it is well known that taking digital photographs from radiology monitors and/or displays for image reading introduces severe image artifacts and/or distortions, which hamper the application of commonly known image pattern searches based on radiology digital photographs. Nevertheless, many radiologists still use smartphones to capture image portions and send them to colleagues for consultation via standard information services.

Disclosure of Invention

It is therefore an object of the present invention to provide alternative apparatus and/or methods enabling intuitive and easy to use identification of at least one medical reference image similar to a current examination image. In particular, it is an object of the present invention to provide alternative apparatus and/or methods for identifying at least one reference image that is generally compatible with existing radiology hardware and software devices.

This object is solved by a method for identifying at least one medical reference image, a corresponding system, a corresponding computer program product and a computer readable storage medium according to the independent claims. Alternative and/or preferred embodiments are the object of the dependent claims.

In the following, the solution according to the invention is described with respect to the claimed device and with respect to the claimed method. Features, advantages, or alternative embodiments described herein may be equally distributed over other claimed objects and vice versa. In other words, the claims solving the method of the present invention may be amended by features described or claimed with respect to the apparatus. In this case, for example, the functional features of the method are implemented by the target unit or element of the device.

Accordingly, a first aspect of the invention relates to a method for identifying at least one medical reference image. The method includes a number of steps.

The first step involves providing a representation image based on a current examination image depicting a body part of a first patient. The second step involves defining a region of interest in the representation image. Another step involves generating a feature signature for at least a region of interest in the representation image. A further step involves comparing the representative image with a plurality of medical images of at least one second patient stored in a medical image database based on the generated feature signature. A further step involves identifying at least one medical image in the database as a medical reference image, the medical reference image providing a similarity to the representative image above a predetermined threshold.

The generating step is performed using a trained machine learning algorithm.

The current examination image is a medical image acquired using a medical imaging modality, wherein the medical imaging modality corresponds to a system for generating or producing a medical image. For example, the medical imaging device may be a computed tomography system, a magnetic resonance system, an angiographic (or C-arm X-ray) system, a positron emission tomography system, or the like. Accordingly, the current examination image may be a computed tomography image, a magnetic resonance image, an angiographic image, a positron emission tomography image, or the like. The current exam image is a medical image that is currently being analyzed and/or evaluated and/or examined by a radiologist for a first patient to infer clinical findings. The current exam image may currently be displayed on a monitor, display screen, light box set up in the radiology department, or the like, so that the radiologist can examine the current exam image and provide a radiology report for the first patient based on the current exam image. The current inspection image may be a digital image and/or an analog image, for example according to the DICOM format or as exposed X-ray film material. The current examination image represents or depicts a body part of a first patient, which should be understood as the patient currently under examination. The depicted body part of the first patient corresponds to a body region, a body (sub-) region, an organ or tissue such as abdomen, chest, lung, neck, heart, head, leg, etc. or at least a part thereof.

The current examination image, in particular in digital form, may be a two-dimensional, three-dimensional or four-dimensional medical image, or provide two and/or three dimensions in space, with or without an additional time dimension.

The representative image corresponds to a digital representation of the current inspection image, i.e. a digital version of the current inspection image. In other words, the representative image depicts the same body part of the first patient as the current examination image. However, a typical feature of the representative image is that it provides a lower image quality than the current inspection image. The lower image quality is preferably caused by artifacts introduced by generating the representative image. The artifacts may include less spatial resolution, less color or grayscale depth or dynamic range, image artifacts such as moire, distortion, pixel grid patterns, illumination non-uniformity artifacts, etc., as compared to the current inspection image.

The representation image may be a two-dimensional or three-dimensional medical image, providing two or three dimensions in space, both spatial dimensions optionally being provided with an additional temporal dimension. Thus, the representative image may be generated by taking a two-dimensional photograph of one current inspection image or by taking many photographs of several current inspection images depicted in a line on a radiation screen or light box. In case a plurality of current inspection images are taken, the representation images may correspond to an image stack, wherein the image stack may comprise three spatial dimensions (three-dimensional volume) or two spatial dimensions and one temporal dimension (video). Here, the providing means may comprise a further step of stacking the images.

Alternatively, the representative image may be generated by taking a digital snapshot of the current medical image depicted on the display screen using a corresponding computer program. Thus, software running on the operating system of the radiology workstation may be used to take or capture snapshots as representative images. In doing so, fewer image artifacts are expected. The resulting representation image does represent the current examination image, i.e. a DICOM image with the original bit-depth, geometry and other acquisition parameters. The representative image may be generated according to a well known graphics or video format, such as a compressed format, e.g., JPEG 2000 or line data format or TIFF or MPEG, AVI, MOV, FLV or RM format.

The generated representative image is used to define a region of interest. A region of interest is to be understood as a set of image elements, e.g. representing pixels or voxels within an image. The region of interest comprises at least one but preferably a plurality of image elements representing an image. The region of interest may likewise comprise all image elements representing the image. The region of interest may be a two-dimensional or a three-dimensional region. The region of interest (ROI) represents a region or volume within the delineated body part of the first patient, which is of particular interest for a radiologist to analyze a current examination image of the first patient. For example, the region of interest is positioned such that it includes or covers suspicious or atypical anatomical structures, such as lesions or calcifications. Preferably, the region of interest covers additional adjacent tissue representing an undoubted area for providing additional contextual information. The region of interest covering not only the lesion but also the surrounding tissue can later be used to represent a true hounsfield-like grey scale normalization of the image or just for the region of interest. The region of interest may have any shape, preferably the region of interest is in circular or quadratic form. Preferably, not only one region of interest (ROI) is defined, but also two or more regions of interest. For example, where the representation image depicts more than one lesion or pathology, they may be considered for subsequent feature extraction with the respective regions of interest.

Alternatively, the region of interest may be defined by providing a representation image. Thus, these two steps can be combined, thereby reducing the amount of image data and the necessary computational power involved in the subsequent invention process.

Visual features representing the image are then analyzed. In this step, a characteristic signature is generated for the representative image. The image analysis step, such as feature extraction, may for example comprise the identification, analysis and/or measurement of objects, local and/or global structures and/or textures present in the representation. The feature signature preferably includes not only one but also a number of features as a sum characterizing the analysis image. Most preferably, the above-mentioned image analysis relates to or better is limited to a region of interest. In other words, the algorithm for image analysis is applied only to represent image elements included in a region of interest in an image. However, the remaining image portion of the representation image may also be analyzed and further image features may be determined which additionally characterize the analyzed representation image.

The generated feature signature may include anatomical features and/or structures, such as the presence of landmarks or the size of an organ or the structure, texture and/or density of the identified tissue or organ. The feature signature may likewise comprise parameters characterizing a color and/or gray scale scheme or contrast features or local gray scale gradients present in the analysis image, preferably in the defined region of interest. However, the identified characteristic signature may also comprise parameters indicating information about the surroundings of the current examination image, which may also be present in the representation image, e.g. information indicating the reading software/application for reading or metadata information in the form of a textual representation, e.g. a texture is typically overlaid as patient ID, acquisition protocol parameters or patient specific information to retrieve additional information useful for retrieving similar cases.

Now, the shape of the representation image is compared to a plurality of medical images of at least one second patient. The at least one second patient corresponds to one or more patients other than the first patient. The plurality of medical images corresponds to a set of individual medical images, each medical image representing or depicting at least partially the same body part as the currently examined image of the at least one second patient. The plurality of medical images may include medical images acquired using a medical imaging modality, such as a computed tomography system, a magnetic resonance tomography system, a C-arm X-ray system, and the like. Preferably, the plurality of medical images comprises medical images acquired using the same medical imaging modality as the current examination image. The current examination image and the plurality of medical images depict, include or show at least one common anatomical structure, organ, tissue, etc. of the first patient and the second patient. The plurality of medical images and/or the current examination image may comprise additional or additional anatomical structures that are not present in the respective image with which it is compared. In other words, the current image and the plurality of medical images may cover different or the same field of view. However, the plurality of medical images preferably comprises a medical image overlaying, depicting or showing a body part of the second patient, which corresponds to or at least overlays the region of interest in the representation image. The plurality of medical images may comprise two-dimensional, three-dimensional or four-dimensional images. Preferably, the plurality of medical images is acquired before the current examination image, most preferably long before the current examination image.

Thus, the plurality of medical images may be stored in a medical image database, which may be located on a central cloud server that can access authorized network services, or on a local hospital server such as a RIS or PACS, etc.

The comparison of the images is based on a feature signature representing the generation of the images.

Thus, the generating step may further comprise the image analysis described above applied to the plurality of medical images stored in the database to generate a feature signature for each of the medical images of the plurality of medical images. In other words, the method of the invention applies the same image analysis algorithm to a plurality of medical images. Here, image analysis, e.g. feature extraction, may likewise comprise the identification, analysis and/or measurement of objects, structures and/or textures, contrasts, etc. present in each of the medical images of the plurality of medical images. However, the image analysis may be applied to all image elements of each medical image, or at least to image blocks identified as corresponding to a field of view representing the image. Since the current examination image may have a different field of view compared to the plurality of medical images, similar features may be located at arbitrary positions in each of the medical images, and lesions/pathologies may occur at different positions between patients, the image analysis of the plurality of medical images as described above being applied to all image elements of the plurality of medical images. The feature signatures for a plurality of medical images preferably comprise not only one but also a number of features as a sum of image regions and/or medical images characterizing the analysis.

According to the invention, the step of generating at least the feature signature is performed using a trained machine learning algorithm. Preferably, the trained machine learning algorithm comprises a neural network, most preferably a convolutional neural network. Neural networks essentially construct, for example, the human brain like biological neural networks. In particular, an artificial neural network includes an input layer and an output layer. It may also include multiple layers between the input layer and the output layer. Each layer comprises at least one, preferably a plurality of nodes. Each node may be understood as a biological processing unit, e.g. a neuron. In other words, each neuron corresponds to an operation applied to input data. The nodes of one layer may be interconnected to the nodes of the other layer by edges or connections, in particular by oriented edges or connections. These edges or connections define the data flow between the network nodes. In particular, edges or connections are provided with parameters, wherein the parameters are usually denoted as "weights". The parameter may adjust the importance of the output of the first node to the input of the second node, where the first node and the second node are connected by an edge. In particular, neural networks may be trained. In particular, training of the neural network is performed according to a "supervised learning" technique based on known input values and output values, wherein the known input values are used as inputs to the neural network, and wherein the respective output values of the neural network are compared with the respective known output values. The artificial neural network learns and adjusts the weights of the individual nodes independently as long as the output values of the last network layer sufficiently correspond to the output values known from the training data. For convolutional neural networks, this technique is also referred to as "deep learning". The terms "neural network" and "artificial neural network" may be used as synonyms. Most preferably, during the training phase, the convolutional neural network is trained to identify different, preferably predetermined types of disease patterns, such as emphysema, honeycomb lung (honey combining), and ground glass opacity (ground grain opacity). Each disease pattern may be characterized by individual visual image features. Thus, during the training phase, neural network learning classifies the extracted feature signatures into at least one of these disease patterns. During training, manual mapping of the respective representative images to the respective disease patterns may be applied.

A first set of neural network layers may be applied to extract features from the image. In this case, the gray scale and/or the color value of the medical image, i.e. of each individual image element of the image, is used as input value for the neural network. The thus extracted features, such as contrast, gradient, texture, density, etc., may be fed as input values to a second set of network layers, also referred to as classifiers, for further assigning objects and/or features to at least one of the extracted features present in the image. However, both functions of the described neural network can equally be performed by separate individual neural networks. In other words, the image analysis for feature extraction may be carried by a first neural network, and classification, i.e. object and/or feature assignment, may be performed by a second neural network.

After image analysis and feature signature generation, the representative image and the plurality of medical images are compared with each other. In other words, the respective feature signatures are compared with each other. For example, each individual feature contributing to the feature signature may be compared separately. Alternatively, the comparison is based on a reduced feature parameter that is based on a plurality of individual features that contribute to the feature signature. Preferably, this step comprises comparing using the neuron activation values of the last but one layer of the neural network as the feature signature representing the image. Thus, based on the comparison, at least one medical image in the database is identified as a medical reference image. The medical reference image is identified or selected according to a similarity to the corrected representative image, in other words based on the distance and/or difference measure of the compared feature signatures.

The predefined threshold may be set and/or set automatically by a user. The threshold value may also depend on the combination of features analyzed and/or identified in the representation image. The threshold may comprise a plurality of individual thresholds, each value belonging to an individual feature of the identified group of features, or only one common threshold taking into account all identified features.

Providing representative images makes the present invention compatible with any radiological apparatus. Thus, not only modern advanced visualization stations can benefit from the automatic reference image search of the present invention, but only small radiological entities, such as light boxes, with old or outdated equipment can apply the described method of the present invention. Image feature comparison using neural networks enables a fast, i.e., substantially rapid (on the flight) search of a large number of medical images stored in a database.

According to a preferred embodiment of the present invention, the step of providing a representative image comprises taking a digital photograph of the current inspection image. In other words, the present invention utilizes a digital camera of a mobile device, such as a smart phone or tablet device. A digital or analog current examination image depicted on a radiation monitor or a radiation lamp box is converted into a presentation image by taking a photograph of the current examination image using a digital camera. The inventors have realized that digital cameras can in principle be used anywhere. Therefore, the present invention can be widely used.

According to another preferred embodiment of the invention, the step of defining the region of interest is based on identifying an anatomical feature in the representation image, wherein the anatomical feature is indicative of a pathological condition of the patient. In this embodiment, the definition or location and/or size of the region of interest is dependent on anatomical features identified in the representation image. The anatomical feature in this embodiment is characterized in that it is indicative of a pathological condition. Pathological conditions include any anatomical deviation from normal and/or average conditions of organs, tissues and anatomical structures and/or only parts thereof. Pathological conditions may include, for example, atypical deformation, growth or contraction of an organ, atypical tissue concentration, calcification, cavities or lesions such as cysts or nodules, and the like. Furthermore, pathological conditions may include prolonged texture and/or structural deviations/changes of organ tissue compared to normal/average states. In other words, in this embodiment, the definition of the region of interest is based on the lesion detection step. Anatomical features indicative of pathological conditions may be detected visually by a radiologist or by automatically applying a lesion detection algorithm.

According to another preferred embodiment of the invention, the step of defining the region of interest is performed manually by a user. This step may be implemented, for example, using a user interface of the mobile device for generating and providing the representation images. The representation may be displayed on a screen of the mobile device for region of interest definition. To this end, a graphical ROI (region of interest) tool comprising a closed outer line may be provided, which may be superimposed to the displayed representation image and may be manipulated via a touch pad of the mobile device. Manipulation of the ROI tool may include positioning and changing the size and/or shape of the closed outer line within the representation image to ensure that the lesion is fully included in the region of interest so defined. All image elements representing the image covered by the ROI tool belong to the region of interest for the following step and preferably consider the feature/object/lesion extraction and/or identification on which the image comparison is based. Alternatively, defining the region of interest may be performed automatically or semi-automatically based on the results of the lesion detection algorithm. Here, the algorithm may suggest regions of interest based on lesion detection or based on default settings, which may then be manually adjusted.

Thus, the present invention ensures that the region of interest covers atypical or abnormal anatomical features. Defining a three-dimensional region of interest may include manually defining the region of interest in more than one image slice of a three-dimensional or video image stack. Alternatively, it may comprise a manual definition in only one image slice and an automatic propagation of the manually defined ROI to the remaining image slices, wherein the automatic propagation may be based on at least one detected object, preferably an object identified as a lesion comprised in the manually defined region of interest.

According to an alternative embodiment, the region of interest is defined by a field of view representing the image defined when the digital photograph was taken.

Artifact correction representing an image may include well-known image processing techniques such as, for example,

-the segmentation of the image is carried out,

-the registration of the images is carried out,

-image filtering in the spatial as well as in the frequency domain,

-the detection of the object(s),

-the subtraction of the object(s),

-whole image subtraction or (at least partial) image superimposition,

-texture, pattern and/or structure analysis and/or removal,

-the contrast is enhanced by a contrast enhancement,

-the reduction of noise is carried out,

-the geometric normalization of the image,

-image grey scale normalization, etc. Artifact correction is used to at least partially reduce or completely eliminate at least one artifact introduced into the representative image during generation. Artifact correction may be applied in two or three spatial dimensions.

According to another preferred embodiment of the invention, the step of correcting the artifacts in the representative image comprises correcting at least one of the following artifacts: image element grid artifacts, dust artifacts, LCD refresh artifacts, lighting artifacts, and artifacts due to limited gray scale dynamic range.

As previously mentioned, taking a photograph of the current inspection image to provide a representative image introduces artifacts into the representative image. When applying the artifact correction algorithm, the following artifacts may preferably be reduced. Image element grid artifacts, preferably pixel grid artifacts, may originate from a current examination image displayed with a screen or monitor providing a certain spatial resolution, wherein the image element grid structure is defined by a plurality of image elements in the x-direction and the y-direction. The mobile device may be held at any distance and/or angle relative to the screen surface, which may cause spatial distortion of the depicted anatomy. Other image element grid artifacts may be caused by defects in the individual display elements of the radiology screen/monitor used to display the current examination image. Additional image element grid artifacts may be generated by the grid structure itself, which may be captured by taking a picture. Image element grid artifacts may be corrected, for example, using fourier transform and frequency filtering techniques. The display screen, as well as the classical radiological light box or medical image film material, may be dusty, thus causing dust particles to also be represented in the representative image. These artifacts may be corrected, for example, using object detection adapted to detect dust particles in the representation image and subtracting the detected objects from the representation image. Furthermore, ambient lighting when taking a photograph for generating a representation image may cause color or grayscale distortions in the representation image, which may result in deviations in image contrast and/or color representation. Where the digital photograph corresponds to a spatially subsampled version of the current inspection image, the moire effect may occur in the representative image. The contrast representing the image may be further compromised when converting the full computed tomography and/or magnetic resonance grey level dynamic range to the rather limited dynamic range of the digital camera by taking a picture. Of course, the list of artifacts that may be introduced to represent the image is not final, and other and/or additional artifacts, such as increased noise levels, may also be taken into account during artifact correction.

In general, such artifact correction is known and can be used for other image processing applications, such as correcting RAW images of digital video cameras. Here, the software program automatically corrects for optical distortion, "vignetting". They are also suitable for noise reduction, for image alignment, etc. In addition to this, it is known that scanned film/video is corrected to eliminate scanning artifacts as dust/scratches.

According to another preferred embodiment of the invention, the step of correcting the artifacts in the representation images is also performed by a trained machine learning algorithm. According to this embodiment, this step may be implemented by applying a machine learning algorithm to the artifact correction. Most preferably, the artifact correction is performed using the same neural network used for comparing the representation image with the plurality of medical images and for identifying the at least one reference image. In particular, the neural network may comprise groups of layers of the representation image for providing artifact correction before performing feature extraction on the corrected representation image. Alternatively, the artifact correction is performed by respective neural networks, which use the original representation image as input values for the first network layer and provide the artifact corrected representation image as output values for the last network layer. The network may be trained using a supervised training approach and providing training data comprising a plurality of pairs of digital photographs, i.e., representative images and corresponding current inspection images. These training data specifically account for image artifacts introduced by capturing photographs from a monitor or light box. Thus, the trained system is able to approximate the original current inspection image by applying appropriate image correction steps to the representation image. Preferably, the training data is synthetically generated by simulating addressed image artifacts. Alternatively, a pair of real images representing the image and the current medical image may be used as training data. One possible way to optimize automatic image artifact correction to produce a corrected representative image that looks like the current inspection image is to use an "antagonistic neural network". One of these networks is trained to produce a "natural" or "pristine" image, i.e., the current inspection image captured from the screen, i.e., the representative image, while the second network is trained to distinguish between a "real" current inspection image and a "simulated" current inspection image. The two networks compete with each other. Their training is repeatedly improved. Thus, a high quality simulator network and a high quality discriminator network are produced.

As an alternative to the active artifact correction and the step of providing an artifact corrected representation image, an "uncorrected" representation image may be used as an input to the neural network. In this case, the neural network is trained to detect and/or analyze image artifacts in the representation images, but not to correct them, but to compensate for them when generating corresponding feature signatures. According to this alternative, the step of correcting image artifacts is essentially interpreted by the neural network.

According to another preferred embodiment of the invention, the step of comparing and identifying is performed taking into account residual artifacts remaining after the representative image is corrected. This embodiment advantageously takes into account that the corrected representation image may still comprise image artifacts and does not show a similar image quality as the original current examination image. Furthermore, the algorithm is adapted to take into account deviations in the signature of features representing the image (corresponding to all features/objects detected and/or analyzed) caused by reduced quality during the image search. This is for example achieved by setting a predefined similarity threshold for the comparison of feature signatures to a lower level. Preferably, the threshold value may be set or adjusted based on image artifacts indicative of the identification and correction of the image.

According to another preferred embodiment of the invention, the method further comprises acquiring imaging parameters of each of the current examination image and the plurality of medical images, the imaging parameters being indicative of an imaging modality used for generating the respective image, and identifying the reference image based on the identified imaging parameters. The representative image may contain information about the medical imaging modality used for image acquisition, e.g., the current examination image may be depicted in DICOM metadata such as X-ray tube voltages representative of CT measurements. Alternatively, the representative image may include textual or semantic information related to the image acquisition as part of the reading software used to render the current inspection image on the screen. Thus, this embodiment further comprises an image analysis step, which also involves Optical Character Recognition (OCR) techniques and/or semantic analysis to infer the applied acquisition technique. By doing so, images identified as reference images may be further filtered and/or classified based on their acquisition techniques.

Alternatively, the acquisition method may be estimated based on image feature analysis, for example, based on image type and/or image contrast, and the reference images may be prioritized according to their image types. Alternatively or additionally, the medical image may be analyzed for a reference anatomical location of the region of interest, for example also by further analyzing surrounding image portions and using the proximity of the anatomical locations for prioritizing the reference images.

Preferably, the at least one identified reference image may be presented to the user. Preferably, the presentation is performed in an order corresponding to the degree of similarity between the representative image and each of the medical images. Preferably, the at least one reference image is presented using a screen of the mobile device that is also used to provide the representation image. Most preferably, in addition to the at least one reference image, the user may be presented with further information regarding a confirmed diagnosis of the associated second patient when presenting the at least one reference image. By doing so, the method provides simple and fast access to expert knowledge and thus enables a profound differential diagnosis of the first patient. Further information on confirming the diagnosis may for example include a list of radiological findings on which the diagnosis is based, personal information on the second patient such as age, weight or sex, information on the patient's medical history, etc. Further information on confirmed diagnosis may be extracted, for example, from medical textbook databases such as "Thieme", or may be inferred from local healthcare enterprise Picture Archiving and Communication Systems (PACS) and/or hospital or radiology information systems (HIS/RIS).

According to a second aspect, the invention relates to a system for identifying at least one medical reference image. The system comprises:

-an interface unit adapted to

-providing a representation image representing a body part of a first patient, and

-a computing unit adapted to

-defining a region of interest in the representation image,

-generating a feature signature at least for a region of interest in the representation image,

-comparing the representation image with a plurality of medical images stored in a medical image database based on the generated feature signatures,

-identifying at least one medical image in the database as a medical reference image, the medical reference image showing a similarity above a predetermined threshold with respect to the representation image,

wherein the computing unit is adapted to run a trained machine learning algorithm for performing the generating step.

The calculation unit may optionally comprise a correction unit adapted to correct at least one artifact in the representation image before generating the feature signature.

The interface unit may be understood as a mobile device comprising a digital camera and a display screen, or alternatively as a workstation comprising a display screen thereof. The interface unit may be adapted to generate the representation image. It also comprises an interface for data exchange with a local server or with a central network server via an internet connection. The interface unit is further adapted to receive at least one reference image and/or additional information related to the second patient via the interface and to display the received information to the user via the display screen.

According to a preferred embodiment of the invention, the system is adapted to implement the inventive method for identifying at least one medical reference image. The calculation unit may comprise a definition unit adapted to define at least one region of interest. The calculation unit may include: an optional correction unit adapted to correct at least one artifact in the representation image; a generation unit adapted to generate a feature signature at least for a region of interest (corrected) representing an image; a comparison unit adapted to compare the representation image with a plurality of medical images stored in a medical image database based on the generated feature signature; and an identification unit adapted to identify at least one medical image in the database as a medical reference image showing a similarity with respect to the (corrected) representation image above a predetermined threshold. The calculation unit is preferably adapted to run a trained machine learning algorithm to perform the generating step.

The computing unit may be implemented as a data processing system or as part of a data processing system. Such data processing systems may include, for example, cloud computing systems, computer networks, computers, tablets, smart phones, and the like. The computing unit may comprise hardware and/or software. The hardware may be, for example, a processor system, a memory system, and combinations thereof. The hardware may be configured by software and/or may be operated by software. In general, all units, sub-units or modules can exchange data with one another at least temporarily, for example via a network connection or a corresponding interface. Thus, the individual units may be positioned separately from each other, in particular the defining unit may be positioned separately from the rest of the computing unit, i.e. at the mobile device.

According to another aspect, the invention relates to a computer program product comprising a program element which, when loaded into the memory of a computing unit, directs the computing unit of the system to identify at least one medical reference image for performing the steps of the method according to the invention.

According to a further aspect, the invention relates to a computer-readable medium, on which a program element is stored, which, when being executed by a computing unit, is readable and executable by the computing unit of the system for identifying at least one medical reference image for carrying out the steps of the method of the invention.

The following advantages are achieved by the computer program product and/or the computer-readable medium: an already existing provisioning system can be easily adapted by software updates to work according to the proposals of the present invention.

The computer program product may be, for example, a computer program or comprise another element alongside a computer program as such. These other elements may be hardware, such as a memory device having a computer program stored thereon, a hardware key for using the computer program, etc., and/or software, such as a document or a software key for using the computer program. The computer program product may also include development materials, runtime systems, and/or databases or libraries. The computer program product may be distributed over several computer instances.

In summary, the invention serves to improve the assistance of radiologists to provide reference images for cases that look similar to the diagnostic proof of the current examination image, and may also provide relevant patient-specific clinical information. European patent application EP18160372.1, which is fully incorporated herein by reference, provides meaningful content for diagnostic decision support, however, the present invention provides this information without integrating or configuring image search into available viewing/reading software applications. The present invention thus enables diagnostic support for radiologists independent of reading devices ranging from conventional light boxes (still often used in e.g. golden brick countries such as in rural areas of china or brazil) to PACS reading workstations from any supplier. Providing a solution to capture images from a radiology reading display on a mobile device and providing fast feedback with similar cases for which diagnosis is known enables AI-based reading assistance to radiologists and clinicians in a very flexible and extensible manner.

Drawings

The above described features, characteristics and advantages of the invention and their implementation will become clearer and more easily understood from the following description and embodiments, which will be described in detail with reference to the accompanying drawings. The following description does not limit the invention to the embodiments contained. In different drawings, the same components or parts may be denoted by the same reference numerals. In general, the drawings are not to scale. In the following drawings:

figure 1 depicts an inventive system for identifying at least one medical reference image according to an embodiment of the invention,

figure 2 depicts an inventive calculation unit for identifying at least one medical reference image according to an embodiment of the present invention,

figure 3 depicts the inventive method for identifying at least one medical reference image according to an embodiment of the invention,

FIG. 4 depicts a neural network that may be applied to at least one embodiment of the present invention.

Detailed Description

Fig. 1 depicts a system 1 for identifying at least one medical reference image REFI according to an embodiment of the invention. The system 1 is adapted to perform the method of the invention according to one or more embodiments, for example, as further described with reference to fig. 3. The system 1 comprises a mobile device 2 in the form of a smart phone. The mobile device 2 comprises an interface unit in the form of a digital camera with a corresponding lens and optical sensor system. The system also includes a radiology workplace screen or light box 3 that depicts the current exam image CURI. The mobile device 2 is adapted to take a digital photograph of the displayed current examination image CURI, thereby providing a representation image REPI. The mobile device 2 further comprises a display for displaying the representation image and for manually defining the region of interest ROI in the representation image REPI. The mobile device 2 further comprises a processing unit 10 adapted to execute at least one software component, for example in the form of a software application for serving the digital camera, for providing a graphical user interface for defining the region of interest ROI, for displaying the representation image REPI on the display and/or within the graphical user interface and/or for processing the manually input region of interest ROI and the representation image REPI. The system comprises a server system 6, the server system 6 comprising sub-units adapted to correct at least one artifact in the representation image REPI, to generate a feature signature FESI at least for a region of interest ROI of the corrected representation image REPI, to compare the representation image REPI with a plurality of medical images MEDI based on the generated feature signature FESI, and to identify at least one of the medical images MEDI as a medical reference image REFI showing a similarity with respect to the corrected representation image REPI above a predetermined threshold. The server system 6 is adapted to run a trained machine learning algorithm, at least for performing the generating steps as further described with respect to fig. 4. The server system 6 and the mobile device 2 together comprise a computing unit 7 of the invention, as described in further detail in relation to fig. 2. The system 1 further comprises a database 8 for storing a plurality of medical images MEDI. Here, more than 500000 medical images MEDI may be stored. The database 8 may be implemented as cloud storage. Alternatively, the database 8 may be implemented as a local storage or an extended storage, such as a PACS (picture archiving and communication system). The system 1 optionally comprises a further database 9 for storing feature signatures FESI of a plurality of medical images MEDI, i.e. here more than 500000 preprocessed feature signatures may be stored. The medical images MEDI in the database 8 and the feature signatures FESI in the database 9 may be interconnected, for example, via reference markers, such that each feature signature FESI is unambiguously related to one medical image MEDI. The databases 8 and 9 may equally be combined in one database comprising both image data and corresponding feature data. The medical images MEDI in the database 8 may be updated continuously, e.g. daily or weekly, e.g. by a database provider, such as the medical textbook provider "Thieme". In parallel, the database 9 can be updated with a new feature signature FESI once a new medical image MEDI is integrated into the database 8. The database 8 may also store other clinical information related to the medical images MEDI, wherein the clinical information may include, for example, related medical findings, personal information related to the at least one second patient, patient history information, and the like. Alternatively, another database (not shown) may store the medical image-related information.

The server system 6 may comprise a computer/processing unit, a microcontroller or an integrated circuit. Alternatively, the server system 6 may comprise a real or virtual group of computers, for example a so-called "cluster" or "cloud". The server system may be a central server, such as a cloud server, or a local server, such as located at a hospital or radiology site.

The individual components of the inventive system 1 can be connected to one another at least temporarily for data transmission and/or exchange. The mobile device 2 communicates with the server system 6 via an interface to transmit, for example, a representation image REPI for further processing, a defined region of interest ROI or an identified reference image REFI and optionally relevant clinical information for presentation on the mobile device 2. For example, the server system 6 may be activated on a library of requests, where the requests are sent by the mobile devices 2. The server system 6 also communicates with the databases 8 and 9 via an interface. Here, retrieving the feature signature FESI of the medical image is used for comparing and/or retrieving the medical reference image REFI and optionally relevant clinical information when identifying the medical reference image REFI. It is also possible to activate the database 8 or 9 on the requesting repository, wherein the request is sent by the server system 6.

The interface for data exchange may be implemented as a hardware interface or a software interface, such as a PCI bus, USB, or firewire. The computing or processing unit may comprise a hardware or software component, such as a microprocessor or FPGA (field programmable gate array). The storage unit, e.g. a database, may be implemented as Random Access Memory (RAM), as durable mass storage (hard disk drive, solid state disk, etc.).

Preferably, the data transfer is implemented using a network connection. The network may be implemented as a Local Area Network (LAN) such as an intranet or a Wide Area Network (WAN). The network connection is preferably wireless, for example as a wireless LAN (WLAN or WiFi). The network may include a combination of different network examples.

Fig. 2 depicts the inventive calculation unit 7 for identifying at least one medical reference image REFI according to an embodiment of the invention. The calculation unit comprises a processing unit 10. The processing unit 10 is located at the mobile device 2 or better a part of the mobile device 2. The processing unit 10 is arranged to execute a software application for serving the digital camera, for providing a graphical user interface for defining the region of interest ROI, for displaying the representation image REPI on the display and/or within the graphical user interface and/or for processing the manually input region of interest ROI and the representation image REPI, and for transmitting them to the server system for further processing. The user may activate the software application via the graphical user interface and may retrieve the software application, for example, by downloading the software application from an internet application store. The calculation unit 7 of the invention also comprises respective calculation (sub) units 11, 12, 13 and 14. The subunit 11 is adapted to process the received representation image REPI to correct at least one artifact. The subunit 11 may be arranged to execute or run a trained machine learning algorithm for performing the step of correcting the representative image REPI. The sub-unit 12 is adapted to process the corrected representation image REPI to generate a feature signature FESI for the entire representation image REPI, alternatively at least for a region of interest ROI in the corrected representation image REPI. The subunit 12 may also be adapted to generate feature signatures in the same way for a plurality of medical images MEDI stored in the database 8. Therefore, the sub-unit 12 is particularly adapted to execute or run a trained machine learning algorithm to perform the step of generating the feature signature FESI. The subunit 13 is adapted to compare the representation image REPI with a plurality of medical images MEDI stored in the medical image database 8 based on the generated feature signature FESI. The subunit 14 is adapted to identify at least one medical image MEDI in the database 8 as a medical reference image REFI showing a similarity with respect to the corrected representation image REPI above a predetermined threshold. Each subunit of the inventive calculation unit 7 may be implemented as a separate subunit or as a physically integrated subunit, e.g. subunits 13 and 14 may be implemented as only one overall comparison and identification subunit. As already described with reference to fig. 1, each subunit may be individually connected to other subunits and/or other components of the inventive system 1, wherein a data exchange is required for performing the inventive method. For example, the subunits 13, 14 may be connected to the databases 8 and/or 9 for retrieving feature signatures FESI of a plurality of medical images MEDI or for retrieving medical images MEDI identified as reference images REFI.

Fig. 3 depicts an inventive method for identifying at least one medical reference image according to an embodiment of the invention. The method comprises several steps. The order of the steps does not necessarily correspond to the numbering of the steps but may also vary between different embodiments of the invention.

A first step S1 relates to providing a medical representation image REPI based on a current examination image CURI depicting a body part of the first patient. The representation image REPI is characterized in that it provides a significantly lower image quality than the current check image CURI. The representation image REPI may be provided by taking a digital photograph of the current medical image CURI using a digital camera of the mobile device. In case the current medical image is a digital image displayed on the screen of the radiological monitor, the representative image may also comprise visual information about the reading software used for displaying the current examination image and also for image analysis, i.e. some color information about the application interface or some text information about the examined/first patient and/or information about the acquisition method used for the current examination image. This information may also be extracted to facilitate later generated feature signatures FESI. A reduced image quality of the image REPI is indicated due to artifacts introduced during the taking of the picture. The possible artifact may be any one of the following group: less spatial resolution, less color or grayscale depth or dynamic range, image artifacts such as moire, distortion, pixel grid patterns, illumination non-uniformity artifacts, and the like, or combinations thereof. Therefore, the quality of the representation image REPI is reduced compared to the current inspection image CURI. The steps provided are not limited to the generation or acquisition of the representation image but may also comprise the display of the representation image REPI and/or the transmission of the reference image REPI to other system components involved in the method of the invention than the mobile device 2.

A second step S2 relates to defining a region of interest ROI in the representation image REPI. The definition of the region of interest is performed manually by the user or may alternatively be performed automatically or semi-automatically. The manual and semi-automatic definition of the region of interest ROI comprises displaying the representation image REPI via a graphical user interface, preferably of the mobile device 2. The user may, for example, position the region of interest tool visualized for the user to overlay to the displayed representation image REPI, also including optional adjustment of the size and/or shape of the tool. Defining the region of interest ROI is based on identifying anatomical features in the representation image, wherein the anatomical features are indicative of a pathological condition of the patient. In other words, the position, size and shape of the region of interest preferably depend on the features representing anomalies or non-representatives present in the image. The abnormal features may correspond to any deviation of anatomical structures, organs or tissues such as lungs, heart, blood vessels, brain, e.g. increased or decreased local tissue density, cysts, calcifications, etc. Thus, the atypical anatomical feature represents a pathological condition of the first patient. The anatomical features may be visually inspected or identified by the radiologist, or may be the result of a feature extraction and/or object detection step optionally included in this step S2. Thus, step S2 may be performed, at least in part, on mobile device 2 or on server system 6. If necessary, a corresponding data exchange is involved in this step.

A third step S3 relates to correcting at least one artifact in the representation image REPI. In this step artefacts as listed above, namely image element grid artefacts, dust artefacts, LCD refresh artefacts (in the case of the representation image REPI being three-dimensional comprising one time dimension), lighting artefacts and artefacts due to the limited greyscale dynamic range of the digital camera, are corrected. Artifact correction may include image segmentation, image registration or mapping to well-known anatomical atlases, image filtering in the spatial as well as frequency domain, object detection, object subtraction, whole image subtraction or image superimposition, local and/or global texture, pattern and/or structure analysis and/or removal, contrast enhancement, noise reduction, and the like. It is preferred that the entire representation image REPI is artifact corrected, even if only the image element values of the region of interest ROI are further processed later on. Alternatively, the artifact correction is applied only to the region of interest ROI. Artifact correction is used to at least partially reduce or completely eliminate at least one artifact. Step S3 is preferably performed on the server system 6. The corrected representative images may be transmitted back to the mobile device 2 for display and/or manually confirmed by the radiologist (in the event that artifact correction provides sufficient image quality) or discarded (in the event that artifact correction does not sufficiently improve image quality). At this point, the method of the present invention may be restarted by providing another representative image, in accordance with step S1. Alternatively, this step may comprise providing the user, via the mobile device, with an optical and/or acoustic alert message indicating that the quality of the representation image REPI is insufficient and suggesting the acquisition of a new representation image REPI.

Preferably, correcting artifacts in the representative image is performed by a trained machine learning algorithm, such as neural network 40, as will be described in further detail with reference to fig. 4.

A fourth step S4 involves generating a feature signature FESI at least for the region of interest ROI defined in the corrected representation image REPI. This step is preferably performed using a trained machine learning algorithm, preferably neural network 40, as will be described in further detail with reference to fig. 4. Step S5 is preferably performed on the server system 6 of the system 1 of the present invention. Most preferably, steps S3 and S4 are performed by the same neural network. The feature signature FESI representing the image REPI is preferably represented by the output value of the last but one layer 45 of the neural network 40. Thus, the neural network performs 40 at least one, preferably more, of the following operations on the representation image REPI: identification, analysis and/or measurement of objects, local and/or global structure and/or texture analysis. Therefore, the feature signature FESI preferably includes a plurality of features as a sum characterizing the analysis image. The image analysis may preferably be limited to a region of interest ROI defined for the representation image REPI. However, also the remaining image portion of the representation image REPI may be analyzed and further image features additionally characterizing the representation image REPI may be determined. Thus, the generated feature signature FESI may comprise landmark information or size/diameter of an organ, structure, texture and/or density parameters of the identified tissue and/or organ. The feature signature FESI may likewise comprise parameters characterizing the color and/or gray scale scheme or contrast features or local gray scale gradients present in the analysis image, preferably in the defined region of interest. However, the generated feature signature FESI may also comprise a parameter indicating information about the surroundings of the current examination image CURI, for example a parameter indicating reading software/application for image reading or a parameter indicating imaging modality information in the form of a textual representation. Thus, step S4 may further include acquiring imaging parameters for each of the current examination image CURI and the plurality of medical images MEDI, the imaging parameters being indicative of an imaging modality used to generate the respective image. And incorporates this information into the feature signature FESI. Thus, step S4 may also include image analysis involving Optical Character Recognition (OCR) techniques and/or semantic analysis to infer the applied acquisition technique.

Alternatively, step S4 may include generating a feature signature FESI for each of the plurality of medical images stored in the database 8, and storing the feature signature FESI in the database 9. Preferably, this step can be performed upstream of the method steps of the invention to reduce the execution time when requesting the display of the reference image REFI.

A fifth step S5 involves comparing the artifact-corrected representation image REPI with a plurality of medical images MEDI of at least one second patient stored in the medical image database 8 based on the generated feature signature FESI. Preferably, this step comprises using the output value of the last but one layer of the neural network as the feature signature FESI representing the image REPI. This feature signature FESI is compared with feature signatures FESI of a plurality of medical images MEDI stored in advance, for example, in the database 9. The feature signatures FESI of the plurality of images may correspond to the feature signatures FESI of the entire medical image or to only a part of the medical image MEDI for the image blocks only. The feature signatures FESI of the medical images MEDI in the database 8 are assigned clinically confirmed disease patterns corresponding to defined feature patterns for e.g. honeycomb lung, frosted shadow or emphysema for lung tissue. The representative image REPI can thus be analyzed for several confirmed diagnoses based on the extracted features. This step comprises at least the exchange of data between the server system 6 and the database 9.

A sixth step S6 involves identifying at least one medical image MEDI in the database 8 as a medical reference image REFI providing a similarity with respect to the corrected representation image REPI that is above a predetermined threshold. Preferably, the similarity to the corrected representation image REPI is calculated using the distance and/or difference measure, preferably the euclidean distance, of the compared feature signature FESI. All medical images MEDI with sufficiently similar feature signatures FESI are identified as reference images REFI. A predefined threshold for sufficient similarity is set automatically and/or by the user. The threshold may preferably depend on a predefined combination of features of the disease pattern. The threshold may be in the form of a relative difference or an absolute difference.

The method of the invention advantageously comprises considering residual artifacts remaining after correcting the representation image REPI for the steps of comparing (S5) and identifying (S6). This advantageously takes into account residual image artifacts in the corrected representation image REPI. The algorithm is therefore adapted to take account of deviations in the feature signature FESI representing the image REPI due to reduced image quality during similar image search. This is for example achieved by setting a predefined similarity threshold for the comparison of feature signatures to a lower level. Preferably, the threshold value may be set or adjusted based on image artifacts indicative of the identification and correction of the image.

Optionally, the method of the invention may comprise a further step for presenting the at least one identified reference image REFI to the user. Preferably, the presentation is performed in an order corresponding to the similarity between the representation image REPI and each of the reference image REFI images. The rendering is preferably performed on the screen of the mobile device 2. This step thus involves data transfer between the server system 6, the database 8 and the mobile device 2. Most preferably, in addition to the at least one reference image REFI, further information about the confirmed diagnosis of the relevant second patient may be presented to the user in parallel with the at least one reference image REFI. By doing so, the method provides simple and fast access to expert knowledge and thus enables a profound differential diagnosis of the first patient.

Fig. 4 depicts a neural network 40 according to an embodiment of the present invention, which may be used to perform the method of the present invention according to fig. 3. Neural network 40 is responsive to a plurality of input nodes x of input layer 41_iThe input value of (1). Applying all nodes of the neural network 40 to generate one or more output values o_j. The neural network 40 of this embodiment adjusts the weight or weighting parameter w of each node based on the training data_i(weights) to learn. Input node x_iIs a corrected picture element value, preferably a pixel value, representing the image REPI. Most preferably, only image element values corresponding to the defined region of interest ROI are fed to the input layer 41. Alternatively, more image element values than the region of interest ROI are used as input. Alternatively, in case the neural network 40 is further adapted to perform the step of artifact correcting the representation image REPI, the image pixel values of the original, i.e. uncorrected, representation image REPI may be fed to the input layer 41. The neural network 40 weights 42 the input values based on a learning process. Output value o of output layer 44 of neural network 40_jMay preferably correspond to the identification and/or confirmation of disease patterns identified in the representation image. For example, the output layer 44 may include four output nodes, each of which represents a disease pattern of four disease patterns. A disease pattern is to be understood as a predefined and preferably clinically confirmed combination of visual image features. In other words, the value o is output₁Can indicate a first disease, output a value o₂Can indicate a second disease, output a value o₃Can indicate a third disease, output a value o_jA fourth disease may be indicated. According to a preferred embodiment, the representation image REPI at least partially depicts the lungs of the first patient, e.g. the representation image REPI corresponds to a lung/chest computed tomography image. Output value o of potential according to predefined disease pattern_jMay include, for example, "alveolar lung," "frosted opacity shadow," emphysema, and the like. According to this embodiment, the neural network 40 is trained during a training phase using training data in the form of labeled image samples for four different disease patterns. The labeling of the training data is preferably performed manually in advance.

The artificial neural network 40 further includes hidden layers 43 each including a plurality of nodes h_j. There may be more than one hidden layer 43, where the output value of one hidden layer is used as input value for a subsequent hidden layer. The respective nodes of the hidden layer 43 perform mathematical operations. Thus, the output value h of the node_jCorresponding to its input value x_iAnd a weighting parameter w_iIs used to generate the non-linear function f. Upon receiving an input value x_iThen, node h_jEach input value x may be performed, for example_iWherein the multiplication utilizes a weighting parameter w_iWeighting, defined as follows:

h_j＝f(∑_ix_i·w_ij)

most preferably, node h_jIs generated as a function f of the node activation, e.g. a sigmoid function or a linear ramp function. Thus, the output value hj propagates through the hidden layer 43 and is finally transmitted to the output layer 44. Here again, each output value h may be calculated from a function f of node activation_jThe sum of the weighted multiplications of (a):

o_j＝f(∑_ih_i·w′_ij)

most preferably, the hidden layer 43 comprises a last hidden layer 45 corresponding to the last but one layer of the neural network 40. Most preferably, the final concealmentOutput value h of layer 45_jThe sum of (a) is used as a feature signature of the representative image REPI of the medical image MEDI search.

The illustrated neural network 40 corresponds to a feed-forward neural network. Thus, all nodes h of the network layers 42, 43, 44, 45_jThe output value of the previous layer is processed as an input value in the form of a weighted sum thereof. However, other embodiments of the neural network type may be applied, such as a feedback network, wherein the input values of the nodes of one layer may be the output values of the nodes of the successive network layers.

The neural network 40 may preferably be trained to recognize disease patterns using methods based on supervised learning. Perfected is a back-propagation method, which can be applied to all embodiments of the invention. During the training phase, the neural network 40 is applied to the training input values to produce corresponding and known output values. Calculating a Mean Square Error (MSE) between the generated output value and the expected output value in an iterative manner to adjust the respective weighting parameter w as long as the deviation between the calculated output value and the expected output value is within a predetermined tolerance_i。

The various embodiments, or aspects and features thereof, may be combined with or interchanged with one another as of interest, without limiting or expanding the scope of the invention. Where applicable, the advantages described in relation to one embodiment of the invention are also advantageous in relation to other embodiments of the invention wherever applicable.