阅读说明：本技术 自动扫描工作流程 (Automatic scanning workflow ) 是由 伊莱安娜·汉库 托马斯·郭-法·福 德斯蒙德·特克-本·杨 阿拉蒂·斯雷库马里 达蒂什·达亚 于 2019-05-29 设计创作，主要内容包括：本发明题为“自动扫描工作流程”。本公开提供了一种在某些具体实施中能够关于采集近实时地评估图像的诊断效用的基于规则或基于深度学习的方法。对应地,在扫描仪上自动实现这种算法实际上将模拟医生自己对图像进行实时评级,并减少不需要的重新扫描和召回的数量。在本发明的一个方面,发现图像的诊断效用不是绝对量度,而是取决于读取放射科医师和扫描指示(即,扫描的目的)。因此,根据读取放射科医师和扫描指示来调节阈值(即,成像体积被认为是良好的概率)可以使得减少重新扫描和召回的数量。(The invention provides an automatic scanning workflow. The present disclosure provides a rule-based or deep learning-based approach that, in some implementations, can evaluate the diagnostic utility of an image in near real-time with respect to acquisition. Correspondingly, automatically implementing such an algorithm on the scanner will actually simulate the physician's own real-time rating of the images and reduce the number of unnecessary rescanning and recalls. In one aspect of the invention, the diagnostic utility of the image is found to be not an absolute measure, but rather to depend on the radiologist and the scanning instructions (i.e., the purpose of the scan). Thus, adjusting the threshold (i.e., the probability that the imaging volume is considered good) based on reading the radiologist and the scan indication may result in a reduction in the number of rescans and recalls.)

1. An image analysis system, comprising:

Circuitry configured to receive and process imaging data acquired by one or more scan components of the imaging system, wherein the circuitry is configured to:

Automatically evaluating the diagnostic utility of one or more images of an imaging sequence based on a diagnostic purpose or a combination of the diagnostic purpose and a designated reviewer; and

Providing a rescan indication upon determining that the one or more images do not meet a threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the designated reviewer.

2. The image analysis system of claim 1, wherein the image analysis system is implemented in a magnetic resonance imaging system.

3. The image analysis system of claim 1, wherein the one or more images are automatically evaluated using a rule-based algorithm or a trained deep learning network.

4. The image analysis system of claim 1, wherein the diagnostic purpose corresponds to a disease state or an anatomical or physiological condition of a patient being evaluated.

5. The image analysis system of claim 1, wherein the designated reviewer comprises a reading radiologist separate from an operator who acquired the one or more images.

6. The image analysis system of claim 1, wherein the circuitry is further configured to:

Receiving input comprising a scan command or the scan command and at least a portion of an electronic medical record;

In response to the input, an imaging protocol is automatically determined that includes at least the imaging sequence.

7. The image analysis system of claim 1, wherein the circuitry is further configured to:

Upon determining that the one or more images do not satisfy the threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the designated reviewer, further providing an alternative imaging protocol for performing the image acquisition during a rescan.

8. the image analysis system of claim 1, wherein the circuitry is further configured to:

Upon determining that the one or more images satisfy the threshold, providing an indication of proceeding with a next imaging sequence.

9. A method for imaging a patient, comprising:

Acquiring one or more images of an imaging sequence using an imaging system;

Providing the one or more images as input to an evaluation algorithm, wherein the evaluation algorithm evaluates the diagnostic utility of the one or more images based on a diagnostic purpose or a combination of the diagnostic purpose and a designated reviewer; and

Providing a rescan indication when the evaluation algorithm determines that the one or more images do not meet a threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the designated reviewer.

10. The method of claim 9, wherein the evaluation algorithm comprises one of a rule-based algorithm or a deep learning network.

11. The method of claim 9, wherein the diagnostic purpose corresponds to a disease state or anatomical or physiological condition of the patient being evaluated.

12. The method of claim 9, wherein the designated reviewer comprises a reviewing physician separate from an operator who acquired the one or more images.

13. The method of claim 9, further comprising:

Receiving a scan command or at least a portion of the scan command and an electronic medical record;

Automatically determining an imaging protocol comprising at least the imaging sequence in response to the scan commands or the scan commands and at least the portion of the electronic medical record.

14. the method of claim 9, further comprising:

Upon determining that the one or more images do not satisfy the threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the designated reviewer, further providing an alternative imaging protocol for performing the image acquisition during a rescan.

15. The method of claim 9, further comprising:

upon determining that the one or more images satisfy the threshold, providing an indication of proceeding with a next imaging sequence.

16. A non-transitory computer-readable medium storing instructions executable by circuitry of an imaging system, the instructions comprising:

Instructions for evaluating the diagnostic utility of one or more images of an imaging sequence acquired by the imaging system based on a diagnostic purpose or a combination of the diagnostic purpose and a designated reviewer; and

instructions to provide a rescan indication upon determining that the one or more images do not satisfy a threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the specified reviewer.

17. the non-transitory computer-readable medium of claim 16, wherein the instructions for evaluating the one or more images comprise a rule-based algorithm or a trained deep learning network.

18. the non-transitory computer readable medium of claim 16, wherein the diagnostic purpose corresponds to a disease state or an anatomical or physiological condition of a patient being evaluated.

19. The non-transitory computer-readable medium of claim 16, wherein the instructions further comprise:

Instructions for receiving input comprising a scan command or the scan command and at least a portion of an electronic medical record;

Instructions for automatically determining an imaging protocol comprising at least the imaging sequence in response to the input.

20. The non-transitory computer-readable medium of claim 16, wherein the instructions further comprise:

instructions for providing an alternative imaging protocol for performing the image acquisition upon determining that the one or more images do not satisfy the threshold established for the diagnostic purpose or for the combination of the diagnostic purpose and the designated reviewer.

Technical Field

The subject matter disclosed herein relates to improving workflow associated with medical imaging or scanning procedures.

background

Non-invasive imaging techniques can obtain images of internal structures or features of a patient or subject. In particular, such non-invasive imaging techniques rely on various physical principles (such as paramagnetic properties of the tissue within the subject, differential transmission of X-ray photons through the imaging volume, gamma ray emission due to differentially distributed radiopharmaceuticals within the body, or acoustic wave reflections occurring through structures within the body) to acquire data and construct images or otherwise represent internal features of the subject.

Certain problems may exist with respect to the workflow associated with imaging scanners. For example, a ready-to-use radiologist may spend a significant amount of time determining an imaging protocol (to be scanned) consisting of a series of imaging acquisitions for each individual patient. That is, it may take time to configure or prescribe an appropriate examination series for the patient. Furthermore, when in fact the currently acquired images are sufficient for diagnostic purposes, the technician may rescan the patient one or more times to acquire better images. Conversely, when the currently acquired images are not sufficient for diagnostic purposes, the technician may not be able to rescan the patient, which may result in the patient having to return to the facility for a second round of imaging. Finally, when artifacts are present or possible in the image, an imaging scheme may be selected that is intolerant to artifact sources that cause the image to contain the artifacts.

Disclosure of Invention

In one embodiment, an image analysis system is provided. According to this embodiment, the imaging system includes circuitry configured to receive and process imaging data acquired by one or more scan components of the imaging system. According to this embodiment, the circuitry is configured to: the diagnostic utility of one or more images of an imaging sequence is automatically assessed based on the diagnostic purpose or a combination of the diagnostic purpose and a designated reviewer. The circuitry is further configured to indicate a need for rescanning upon determining that the one or more images do not meet a threshold established for diagnostic purposes or for a combination of diagnostic purposes and a designated reviewer. Rescanning is defined as requiring repeated imaging acquisitions or performing similar image acquisitions to acquire images of sufficient image quality to provide an accurate diagnosis.

In another embodiment, a method for imaging a patient is provided. According to this embodiment, an imaging system is used to acquire one or more images of an imaging sequence. One or more images are provided as input to an evaluation algorithm. An evaluation algorithm evaluates the diagnostic utility of one or more images based on a diagnostic purpose or a combination of a diagnostic purpose and a designated reviewer. A rescan indication is provided when the evaluation algorithm determines that one or more images do not meet a threshold established for diagnostic purposes or for a combination of diagnostic purposes and a designated reviewer.

In yet another embodiment, a non-transitory computer readable medium storing instructions executable by circuitry of an imaging system is provided. The instructions include: instructions for evaluating the diagnostic utility of one or more images of an imaging sequence acquired by an imaging system based on a diagnostic purpose or a combination of a diagnostic purpose and a designated reviewer; and instructions to provide a rescan indication upon determining that the one or more images do not meet the established threshold for diagnostic purposes or for a combination of diagnostic purposes and a designated reviewer.

drawings

These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

Fig. 1 illustrates an embodiment of a Magnetic Resonance Imaging (MRI) system according to an aspect of the present disclosure;

FIG. 2 depicts a conventional scanning workflow;

FIG. 3 depicts a scanning workflow in accordance with aspects of the present disclosure;

FIG. 4 depicts a convolutional neural network architecture, in accordance with aspects of the present disclosure;

FIG. 5 shows a matrix of recorded Cohen's kappa scores and the number of series the assessor agrees on the diagnostic utility of the image, assuming the series is scanned to exclude MS, in accordance with aspects of the disclosure; and is

FIG. 6 shows a matrix of recorded Cohen's kappa scores and the number of series the assessor agrees on the diagnostic utility of the image, assuming the series is scanned to exclude stroke, in accordance with aspects of the present disclosure.

Detailed Description

one or more specific embodiments will be described below. In an effort to provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.

When introducing elements of various embodiments of the present embodiments, the articles "a," "an," "the," and "said" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Further, any numerical examples in the following discussion are intended to be non-limiting, and thus additional numbers, ranges, and percentages are within the scope of the disclosed embodiments.

The high resolution and versatility of Magnetic Resonance Imaging (MRI) contrast makes it a suitable choice for diagnosing neurological disorders. In the case of acquiring a diagnostic set or series of images, the imaging series is typically repeated when the scan technician determines that the diagnostic utility of one or more images of the respective series is insufficient to make the intended diagnosis due to reasons such as patient motion, metal artifacts, incorrect field of view positioning, hardware malfunctions, and the like. While less documented, the opposite problem exists in that technicians assess the diagnostic utility of images sufficiently for the intended diagnostic purpose, the patient is sent home and then recalled for rescanning because the radiologist cannot use the acquired images for diagnosis. By ensuring that only the series that needs to be repeated is repeated and no one is sent home after acquiring images of insufficient diagnostic utility, it is important to correctly adjust or optimize the number of rescans to optimize the efficiency of the healthcare system. However, this problem is not easily solved because it is the radiologist that ultimately determines whether the diagnostic utility of the image is sufficient, and it is the scanning technician that makes the rescan determination at the time of imaging. Reading radiologists generally does not suggest the diagnostic utility of the image at the time of image acquisition (i.e., before the examination is complete). In addition, radiologists may express different opinions when evaluating diagnostic utility or diagnosing disease. That is, a given image may have sufficient diagnostic utility for one physician but insufficient diagnostic utility for another physician. Note that when the number of rescans is correctly adjusted, it means that rescanning is performed only for the effective clinical purpose of having information missing or the acquired images not having sufficient image quality or diagnostic utility to provide an accurate clinical diagnosis.

In accordance with the present disclosure, a method for simplifying scanner workflow associated with image acquisition is provided. In one aspect, the imaging protocol may be automatically decided based on scan commands from a physician or a combination of scan commands in conjunction with a patient (electronic) medical record. In a conventional workflow, determining an imaging protocol can be a time consuming process in which a radiologist on-standby decides the imaging protocol for each examination to be scanned. Additionally, in one implementation, as the examination begins and an operator (e.g., a technician) of the imaging system scans each series of images, an automated process determines whether the diagnostic utility of the acquired images is sufficient to diagnose the suspected disease that the patient was re-diagnosed with scanning. Since different radiologists have different requirements on image defects and/or artifacts, the evaluation can be done generally taking into account the reason for scanning the patient (i.e. for diagnostic purposes), but without information on who will read the examination. Such diagnostic purposes may correspond to, for example, a disease state or anatomical or physiological condition of the patient being evaluated. Alternatively, information about the person reading the radiologist for the respective scan may be used to reduce rescanning and recalls while still providing the physician with sufficient information to make a diagnosis. If the diagnostic utility of the image is sufficient for a given indication, the examination will proceed according to the initial imaging protocol, e.g., an indication may be provided to the operator to proceed with the next sequence (i.e., a continuation indication). If the diagnostic utility of the image is deemed insufficient, any sources of artifact may be identified and an indication of a repeated imaging sequence (i.e., a rescan indication) may be provided to the technician, including in some implementations a suggestion of an alternative imaging scheme for addressing the artifact. By way of example, such alternative imaging schemes may produce comparable image contrast, but are less sensitive to sources of artifacts identified in previous scans (e.g., patient motion, metal artifacts, incorrect field of view positioning, hardware failures, low SNR, etc.).

in view of the foregoing, in certain implementations, the present disclosure provides rule-based or deep learning-based methods that can assess the diagnostic utility of an image (regardless of contrast, anatomy, or orientation) in near real-time. The diagnostic utility may be automatically assessed in this way based on one or both of the diagnostic purpose of the image and/or the knowledge of the reading physician. As used herein, diagnostic purposes correspond to a disease state or anatomical or physiological condition of a patient being evaluated. In particular, in one implementation, the performance of such algorithms may be verified against the ratings of multiple scan technicians and radiologists and according to scan instructions (i.e., diagnostic purposes for acquiring images). Although there is a wide divergence in the diagnostic utility ratings of different radiologists in images with low or medium levels of artifact, the consistency between the determination algorithm and each physician in studies supporting the present disclosure may be significantly higher (as evidenced by Cohen's kappa) than multiple radiologists when assessing the same image set. Accordingly, automatically implementing such an algorithm on the scanner effectively simulates the physician's own real-time rating of the images and reduces the number of unnecessary rescanning and recalls. Thus, as discussed herein, the diagnostic utility of an image is not an absolute measure, but rather depends on the reading radiologist and the scan indication (i.e., the diagnostic purpose of the scan). Thus, adjusting the threshold (i.e., the probability that the imaged volume is considered good) based on ranking physician and scan indications may result in a reduction in the number of rescans and recalls. According to this approach, these two variables can be provided to the scanner console in a manner similar to the way that prescription doctor information is currently provided.

Although an MRI example is discussed herein, it should be understood that the invention may be similarly implemented using different types of image datasets, including datasets acquired using other MRI imaging schemes and/or other imaging modality types and schemes, including Computed Tomography (CT), tomosynthesis, mammography, ultrasound, Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), etc. Thus, the present disclosure is not limited to MRI scan workflows or images, but may also be implemented with respect to other image modalities.

The workflows described herein may be performed by an imaging system (e.g., a Magnetic Resonance Imaging (MRI) system) in which a particular imaging routine (e.g., a diffusion MRI sequence) is managed by a technician. Thus, the imaging system may perform data acquisition and data/image reconstruction at the direction of a technician. Accordingly, to provide context for an MRI example of the present invention, an MRI system 10 is depicted in FIG. 1. It will be appreciated that other imaging modalities may have corresponding functional components and modules, such as detection components, command and control circuitry, data acquisition components, data processing and visualization components, and the like. The magnetic resonance imaging system 10 is schematically illustrated by an example of such a general purpose imaging system and is depicted as including a scanner 12, scanner control circuitry 14, and system control circuitry 16. According to embodiments described herein, the MRI system 10 is generally configured to perform MR imaging.

The system 10 also includes a remote access and storage system or device, such as a Picture Archiving and Communication System (PACS)18, or other device, such as a remote radiology device, so that data acquired by the system 10 may be accessed on-site or off-site. In this manner, MR data may be acquired, such as by a physician, and then processed and evaluated, either onsite or offsite, as discussed herein. While the MRI system 10 may include any suitable scanner or detector, in the illustrated embodiment, the system 10 includes a whole-body scanner 12 having a housing 20 through which an aperture 22 is formed. The table 24 is movable into the aperture 22 to allow a patient 26 to be positioned therein to image a selected anatomical structure within the patient.

The scanner 12 includes a series of associated coils for generating controlled magnetic fields for exciting gyromagnetic materials within the anatomy of the subject being imaged. In particular, main magnet coils 28 are provided for generating a main magnetic field B0 generally aligned with the bore 22. A series of gradient coils 30, 32 and 34 allow the generation of controlled magnetic gradient fields for positionally encoding specific gyromagnetic nuclei within the body of the patient 26 during an examination sequence. A Radio Frequency (RF) coil 36 is configured to generate radio frequency pulses for exciting specific gyromagnetic nuclei within the patient. In addition to the coils attributed to the scanner 12, the system 10 also includes a set of receive coils 38 (e.g., a coil array) configured to be placed (e.g., abutted) proximal to the patient 26. For example, the receive coils 38 may include a neck/chest/waist (CTL) coil, a head coil, a unilateral spine coil, and the like. Generally, the receive coil 38 is placed near or at the top of the patient 26 in order to receive the weak RF signal (weaker relative to the transmit pulse generated by the scanner coil) generated by certain gyromagnetic nuclei within the patient 26 as they return to a relaxed state.

The various coils of system 10 are controlled by external circuitry to generate the required fields and pulses and to read the emission from the gyromagnetic material in a controlled manner. In the illustrated embodiment, a main power source 40 provides power to the primary field coils 28. The drive circuit 42 provides power to pulse the gradient field coils 30, 32 and 34. Such circuitry may include amplification and control circuitry for providing current to the coil, the current being defined by a sequence of digitized pulses output by the scanner control circuitry 14, which may be a diffusion imaging module in one embodiment. Another control circuit 44 is provided for regulating the operation of the RF coil 36. The circuit 44 includes switching means for alternating between active and passive modes of operation in which the RF coil 36 transmits and does not transmit signals, respectively. The circuitry 44 also includes amplification circuitry configured to generate the RF pulses. Similarly, the receive coil 38 is connected to a switch 46 that is capable of switching the receive coil 38 between receive and non-receive modes. Thus, the receive coils 38 resonate with the RF signals generated from the relaxed gyromagnetic nuclei within the patient 26 in the receive mode, and they do not resonate with the RF energy from the transmit coils (i.e., coils 36) to prevent undesired operation in the non-receive mode. Additionally, the receive circuitry 48 is configured to receive data detected by the receive coils 38, and may include one or more multiplexing and/or amplification circuits.

It should be noted that although the scanner 12 and control/amplification circuitry described above are shown coupled by a single line, many such lines may be present in a practical example. For example, a single wire may be used for control, data communication, and the like. In addition, appropriate hardware may be provided along each type of line to properly process the data. Indeed, various filters, digitizers, and processors may be disposed between the scanner and either or both of the scanner and system control circuits 14, 16. By way of non-limiting example, certain control and analysis circuits described in detail below, while shown as a single unit, include additional hardware, such as image reconstruction hardware configured to perform data processing.

As shown, the scanner control circuitry 14 includes interface circuitry 50 that outputs signals for driving the gradient field coils and the RF coils, and for receiving data indicative of the magnetic resonance signals generated during an examination sequence. The interface circuit 50 is coupled to a control and analysis circuit 52. The control and analysis circuit 52 executes commands for the drive circuit 42 and the circuit 44 based on the defined imaging scheme selected via the system control circuit 16. The control and analysis circuitry 52 is also operative to receive the magnetic resonance signals and perform subsequent processing prior to transmission of the data to the system control circuitry 16. The scanner control circuitry 14 also includes one or more memory circuits 54 that store configuration parameters, pulse sequence descriptions, inspection results, and the like during operation. For example, code or routines may be stored in the memory circuit 54 and executed by the control circuit 14 as part of implementing aspects of the present disclosure.

Interface circuitry 56 is coupled to control and analysis circuitry 52 for exchanging data between scanner control circuitry 14 and system control circuitry 16. In certain embodiments, the control and analysis circuitry 52, although shown as a single unit, may comprise one or more hardware devices. The system control circuitry 16 includes interface circuitry 58 that receives data from the scanner control circuitry 14 and transmits data and commands back to the scanner control circuitry 14. The interface circuit 58 is coupled to a control and analysis circuit 60, which may include a CPU or other microprocessor architecture that may be present in a general purpose or special purpose computer or workstation. The control and analysis circuitry 60 is coupled to memory circuitry 62 to store program code for operation of the MRI system 10 and to store processed image data for later reconstruction, display and transmission. The program code may execute one or more algorithms that, when executed by the processor, are configured to perform reconstruction of the acquired data, and may also include algorithms for generating images. For example, code or routines may be stored in the memory circuit 62 and executed by the control and analysis circuit 60 as part of implementing aspects of the present disclosure.

additional interface circuitry 64 may be provided for exchanging image data, configuration parameters, etc. with external system components, such as the remote access and storage device 18. Finally, the system control and analysis circuitry 60 may include various peripherals to facilitate operator interface and to generate a hard copy of the reconstructed image. In the illustrated embodiment, these peripheral devices include a printer 66, a monitor 68, and a user interface 70, which includes devices such as a keyboard or mouse.

It should be noted that the described MRI system is provided as an example only. Other types of MRI systems (e.g., "open" MRI systems) as well as other imaging modalities may be used with the present invention.

In view of the foregoing, the following problems may arise in the context of a scanner workflow for imaging a patient using an imaging system, such as the MRI system described above.

(1)On-demand radiologists spend a significant amount of time determining the imaging to be scanned for each individual patient Protocol (which consists of a series of imaging acquisitions). By taking as input the scan commands from the radiologist and examining/referencing the patient's medical history, the radiologist decides which series or type of acquisition should be scanned in each examination and for each patient.

(2) When the diagnostic utility is sufficient for diagnostic purposes (e.g., false negative assessments have sufficient image quality), it is important Novel scanning imaging series. If there is some artifact in the image, the technician performing the scan must decide whether the diagnostic utility is sufficient for clinical diagnosis depending on the diagnostic purpose of the examination or whether the scan or imaging sequence should be repeated. In many cases, the technician selects to repeat a series whose diagnostic utility is actually sufficient for diagnostic purposes, thereby unnecessarily lengthening the imaging exam and incurring unnecessary expense for the healthcare system.

b3) When diagnostic utility is insufficient for diagnostic purposes (e.g., false positive assessments have sufficient picture)image quality) of the image data is obtained, Scanning without rescanning. As described above, if artifacts are present in the image, the technician may decide that the diagnostic utility is sufficient for diagnostic purposes and decide to continue the examination without repeating a given imaging series. If the reading radiologist later decides that the diagnostic utility is not in fact sufficient, the patient must be recalled for a second imaging scan, delaying the diagnosis, causing great inconvenience to the patient, and increasing the overall cost of the examination (repeating the series when the quality is deemed insufficient takes much less time than scheduling an entire new examination for the patient).

(4) Is not limited toSelectingImaging scheme tolerant to current artifact sources. While in some cases the technician is sufficiently deciding that the series should be repeated, he or she may repeat the scan with the same initial type of imaging sequence or an alternative form that cannot tolerate sources of artifacts present in the image, such as motion artifacts or metal artifacts.

In view of the foregoing, the present disclosure is directed to a simplified scanner workflow suitable for use with MRI or other imaging modality acquisition. In a conventional workflow, determining an imaging protocol can be a time consuming process in which a radiologist on-standby decides the protocol for each examination to be scanned. In one aspect of the disclosure, the imaging protocol is instead automatically decided based on: (1) scan orders or prescriptions, or (2) scan orders in conjunction with patient (electronic) medical records.

In another aspect, as the exam begins and the technician scans each series, an automated process determines whether the diagnostic utility of the image is sufficient to diagnose the suspected disease that the patient was re-diagnosed to scan. This assessment can be done generally considering the reason for scanning the patient (i.e., scan order), but without information on who will read the examination, since different radiologists have different tolerances for image defects and/or artifacts. Alternatively, information about the person reading the radiologist for the respective scan may be used to reduce rescanning and recalls while still providing the physician with sufficient information to make a diagnosis. If diagnostic utility is sufficient in a given indicated imaging series, the examination will proceed according to the initial imaging protocol.

In yet another aspect, if the diagnostic utility is deemed insufficient, the artifact source will be identified and a recommendation of an alternative imaging scheme for addressing the artifact may be provided to the technician. By way of example, alternative imaging schemes may provide comparable image contrast, but are less sensitive to artifact sources (e.g., motion, metal artifacts, low SNR, etc.) identified in previous scans. In certain aspects, the imaging scheme may be automatically adjusted or parameterized, such as using rule-based or depth learning methods, to account for artifacts.

Accordingly, aspects of the present disclosure may be understood as employing some automated algorithms that perform one or more of the following: (1) simplifying and/or automating imaging protocol determination, thereby reducing or eliminating the time spent by a reading radiologist deciding which series to scan; (2) automatically determining whether the diagnostic utility of the acquired images is sufficient for a given scan indication (e.g., diagnostic purposes) and/or reading by a radiologist; and/or (3) automatically suggest alternative imaging schemes or re-parameterize existing imaging schemes that are less sensitive to artifact sources.

With this in mind, fig. 2 depicts a conventional imaging scanner workflow. According to the workflow, scanning instructions 90 from a radiologist (e.g., instructions or prescriptions to screen for multiple sclerosis in the depicted example) are provided to a radiologist (e.g., a radiologist on-hold) to determine (block 92) an imaging protocol (e.g., a scanning sequence 98) that defines a series of images of a patient to be acquired. The radiologist may also rely on medical records (e.g., electronic medical records 94) having data regarding the patient's medical history and/or characteristics to determine the imaging protocol. In the depicted conventional method, the technician subjectively determines 100 whether the image obtained according to the imaging protocol is of sufficient quality. In conventional approaches, the determination 100 made by the technician may be largely or entirely independent of the diagnostic purpose underlying the scan command 90 (i.e., the diagnosis to be made based on the acquired images) and/or the radiologist that will read the acquired images. Thus, unnecessary rescanning and patient recalls may occur. If the technician determines that the acquired image is of insufficient quality, the imaging sequence 98 is repeated (i.e., rescanned) using the same imaging scheme or alternative scheme (such as accounting for artifacts in the image) at the discretion of the technician. If the technician determines that the acquired images are of sufficient quality, the technician proceeds to the next image sequence 102 or ends the scanning operation.

In contrast, fig. 3 depicts an imaging scanner workflow employing some or all of the above algorithms. The process may be initiated based on the scan command 90 or a combination of the scan command 90 and the electronic medical record 94, as in conventional approaches. In contrast to ready-to-arm radiologists that generate imaging protocols from these materials, a rule-based or trained deep learning algorithm may receive these materials as input and automatically generate imaging protocols (e.g., sequence 98) (represented as "auto protocol" 108 in fig. 3). As in conventional approaches, the technician may acquire images based on an imaging protocol (i.e., sequence 98). Unlike conventional methods, in some implementations of the invention, the determination 110 is an automatic determination (using rule or depth learning based methods) of whether the image obtained according to the imaging protocol is of sufficient quality. In such embodiments, and unlike conventional methods, the determination 110 may be based on one or both of the scan indication (i.e., the diagnostic purpose of the scan) and/or the radiologist who is to view the image. Thus, unnecessary rescanning and patient recalls may be reduced. If the automated algorithm determines that the acquired image quality is insufficient, in one embodiment, another automated algorithm may determine the cause of the image quality insufficiency for diagnostic purposes and may determine or suggest changing the imaging protocol for rescanning. In this way, the likelihood of obtaining a suitable image is increased, such as by using a scanning imaging scheme that can account for image artifact sources. Conversely, if the automated decision process determines that the acquired images are of sufficient quality, the technician proceeds to the next image sequence 102 or ends the scanning operation.

In view of the foregoing and the flow illustrated in fig. 3, the technical benefits of the present disclosure can be understood to include some of the following: (1) reducing the time that a radiologist may spend determining an imaging protocol for an imaging exam for a given patient or removing the task entirely from a ready-to-do list of radiologists; (2) the number of rescans and recalls is reduced (i.e., if the diagnostic utility of the images is deemed insufficient by consideration of only the diagnostic purpose of the scan or unbiased evaluation (e.g., automated and/or algorithmic evaluation) in conjunction with reading the radiologist's information, only the sequence of images will be rescanned), and/or (3) a new imaging protocol is selected or the current imaging protocol is modified for rescanning, thereby mitigating artifacts identified in the initial series of images. These three aspects are discussed in more detail below.

with regard to automatic determination of the imaging protocol, in one example, physician scan orders are converted or otherwise used with patient medical records to automatically generate a scan (imaging) protocol. This can be done in an automated fashion using a deterministic set of rules or through a machine learning algorithm.

With respect to imaging protocol adjustment or re-parameterization for rescanning, once the diagnostic utility of the image is deemed insufficient by an upstream algorithm, another algorithm (e.g., a rule-based or machine learning-based algorithm) may suggest an alternative series to be scanned that is less sensitive to the artifact sources identified above.

In connection with deciding whether the diagnostic utility of an image or series of images is sufficient to diagnose a condition, one or more algorithms may be employed by a general or given radiologist as part of this determination. As discussed herein, the answer to whether a given image or series of images has sufficient diagnostic utility will depend on the indication that the patient is being scanned (i.e., for diagnostic purposes), and in some cases, on who the reading radiologist is. In view of this, in the present disclosure, two separate methods are contemplated to assess diagnostic utility. In certain embodiments described herein, a two-dimensional image is provided as input and a probability that the image is good (i.e., P (good)) (i.e., has diagnostic utility for general or specific reading physicians) is provided as output. Once the ratings are generated for all images in the imaged volume, a per-volume rating may be generated accordingly based on the collective rating. In one implementation, this may be obtained by an arithmetic or geometric mean of the probabilities for each individual slice. The per-volume rating is then compared to a threshold, which may be a function of scan indication or scan indication and reading of the radiologist. For example, volumes with significantly varying ratings from slice to slice are often rated as poor. With sufficient training data for the underlying algorithm, a method can be implemented that uses the volumetric data set (or a portion of the volumetric data set) as input instead of a single two-dimensional slice.

With respect to both envisaged methods, in the first algorithm-based method, deterministic features are extracted from a single image or from the header of the file. This may include, but is not limited to, image features that characterize signals, noise, image focus, edges, texture, and the like. In the MRI context, features from the file's header, including contrast type (T1/T2), echo Time (TE), repetition Time (TR), field strength, etc., may also be included. A Support Vector Machine (SVM) performs classification, thereby outputting a probability (P (good)) that an image is good. In the embodiments discussed herein, the algorithm will be referred to as an SVM.

In the second algorithmic approach, a Convolutional Neural Network (CNN) uses the same single slice image for feature extraction, followed by a fully connected neural network for classification. In the embodiments discussed below, this algorithm will be referred to as Deep Learning (DL).

In view of the foregoing, a study was conducted to evaluate the deep learning approach currently contemplated. The study trained a deep learning algorithm using retrospective data in the form of continuous brain data from patients scanned on three 1.5T scanners, as described herein. By stopping the accumulation of non-duplicate exams and then including only the duplicate subsequent series, data is intentionally enriched in exams that contain at least one duplicate series. Imaging series that repeat for any reason were initially accepted in the study. It was found that the main cause of rescanning the brain series was patient motion (about 95%). The remainder of the repeated sequence is due to low signal-to-noise ratio (SNR) or metal artifacts. Due to this natural data isolation, only motion-corrupted data sets are included in CNN training and testing. Anatomical images of all orientations (sagittal, axial and coronal), all contrast types (proton density, T1, T2, T2 and FLAIR) and all pathologies were included. Diffusion weighted images are not included. Detecting motion in such images is quite simple due to the inherent phase fringe pattern that occurs in a moving patient.

The data is initially evaluated by a single reader, which classifies the images into three categories: 1 (clinically good (CG)), 2 (suspect) and 3 (clinically bad (CB)). The "suspect" data (initially rated as class 2) is sent to another radiologist, who re-rates the images into CG (no repeat) and CB (repeat) categories. Scanning instructions (e.g., Multiple Sclerosis (MS)) are also provided to the second radiologist. The image covers all scan planes, age and pathology. In general, 9554 images belonging to the clinically good group and 7783 images belonging to the clinically poor group were used to train the deep learning algorithm. The purpose of this 2-tier rating is to reserve datasets that are rich in suspect datasets (i.e., category 2) for testing and further MRI technician/radiologist rating, since it is of particular interest to make the correct rescanning decision for these types of datasets.

The entire deep learning based workflow is depicted in fig. 4 with respect to the classification of the deep learning based tested in this study. In this example, a 2D classification model with a tensrflow back end is trained to classify MRI slices as good (i.e., Classification Good (CG)) or bad (i.e., Classification Bad (CB)). The architecture of the Convolutional Neural Network (CNN) is shown in fig. 4. In the depicted example, the deep learning architecture has seven convolutional layers 110, four max pooling layers 112, and three bulk normalization layers 114, although other layer configurations and numbers may alternatively be employed. The dots between the layers labeled max _ pooling _7 and flatten _1 represent another box containing the 7 th 2D convolutional layer, the 7 th bulk normalization layer, and the 8 th max _ pooling layer.

the activation function is a nonlinear exponential linear unit that helps to learn complex patterns from the data. In this example, to enhance the main feature, two merging layers 116 are introduced using multiplication operations. At the end of all convolutional layers, a "flattening" layer 120 is employed, which converts the feature tensor from the convolutional layers to a 1-D tensor. The "tanh" activation 122 is followed by a fully connected layer and a "softmax" output 124. Fully connected layers further help to learn the non-linear combination of features provided by CNN layers. The Softmax function provides probabilities for each class, where the sum of the probabilities equals 1. In this example, class cross entropy is used as the loss function and the optimizer is set to "rmsprop". For purposes of the study discussed herein, images for training and testing are converted to 128 × 128 sizes, and pixel values are converted to z-fraction maps (definition)Is (pixel)_{value of}Mean (series))/standard deviation (series).

The deep learning model as shown in this example outputs a probability for each slice belonging to the CG class. In practice, rescanning decisions are usually made on a per series basis rather than per slice basis. Thus, in this example, the individual slice ratings are combined to compute each series of scores, expressed as the geometric mean of the probability per slice: (a)where P1, P2, … Pn are predictions of slices 1,2, … n). Finally, if P (series) is greater than or equal to a given threshold t, the series is rated CG, and if P (series) is less than the same threshold t, the series is rated CB.

In the initial dataset, 49 series (1344 images of all orientations, contrast types and pathology) not included in the training were reserved for deep learning classification testing. The series was also sent to five radiologists and four MRI technicians. The data set consists mainly of images with low or medium artifact levels: of the 49 series, 5 were initially rated as bad, 6 were initially rated as good, and 38 were initially rated as suspect. Assume that the patient is scanned as: a) exclusion of stroke and b) exclusion of Multiple Sclerosis (MS), all nine survey participants were asked to rate the images as either CG (no rescanning required) or CB (rescanning required). Stroke is an indication that lower image quality is generally required, while MS is an indication that higher image quality is required.

It is observed that doctors will change their diagnostic utility rating for one or more images depending on the purpose of the scan significantly more frequently than technicians (doctor: 36%, technician: 11%). Table 1 below depicts the percentage of ratings that differ based on indication (stroke versus MS), where D1-D5 represent physicians and T1-T4 represent technicians. Thus, the scan command or keywords from the command (i.e., the diagnostic purpose of the scan) may be used as input to an algorithm in order to generate a meaningful rating. That is, without scan indication or diagnostic purposes, the notion of sufficient or insufficient diagnostic utility may not be meaningful.

TABLE 1

A second finding is that there is relatively limited agreement between physicians and technicians as to whether a given series has sufficient or insufficient diagnostic utility. A summary of the number of quality deficiency series found in the 49 data sets sent out is presented in table 2 below. Which presents the results of the technician/radiologist survey. In this table, the number of series deemed to be insufficient image quality for diagnosis (i.e., requiring rescanning) is shown in terms of scan instructions, reading radiologists (D1-D5), and scan technicians (T1-T4).

TABLE 2

It should be noted that if there is no automated algorithm to decide whether the diagnostic utility is sufficient, there will be a large number of rescans and recalls. For example, if the technician T2 is scanning a patient's stroke instructions, whose scan will be read by the physician D1, he or she will rescan 26(═ 28-2) unnecessary series. Alternatively, if technician T3 is scanning for Multiple Sclerosis (MS) indications of a patient whose scan is to be read by doctor D2, he or she will return home to 23 patients (35-12) who need a repeated series, resulting in 23 unwanted recalls.

with respect to table 2, certain observations may be made or repeated. First, doctors have different tolerances for artifacts. D1 (and to some extent D3) typically requires lower subjective image quality of images than D2, D4, and D5 to perform a diagnosis. In many years of experience, artifact tolerant/intolerant physicians are not statistically different from other physicians. Second, the radiologist has a refined view of the diagnostic utility of the image according to the scan instructions (i.e., the diagnostic purpose of the scan). They varied their diagnostic utility rating in 36% of the cases visited according to the scan indication (stroke or MS). In contrast, MRI technicians change their assessments only in 11% of cases. Third, depending on who scans the checks and who reads the checks, there may be a number of rescans or recalls that are not needed. For example, assuming that the patient is screened to exclude stroke and T2 is scanning to read radiologist D1, 26(═ 28-2), an unnecessary rescan (in 49 scan series) will be performed. Conversely, if 49 patients were screened to exclude MS, and if the technician T3 is scanning to read radiologists D2, 23(═ 35-12), the patients (of 49) would be sent home because of insufficient image quality.

Physicians appear to be more or less tolerant of image artifacts in view of the survey demonstrated rating variability shown in table 2. Thus, after deep learning classification (probability that results in sufficient Diagnostic Utility (DU) for each slice), the volumetric probabilities (calculated as the geometric mean of the slice probabilities) are examined against 3 different thresholds (P (good) ═ 0.1, 0.5, and 0.8) to determine the DU rating for a given volume. The results of this comparison are shown in fig. 5.

The table of fig. 5 summarizes the consistency between radiologists, between radiologists and MRI technicians, and between trained deep learning algorithms and radiologists by showing Cohen's kappa, assuming that patients are screened to exclude MS. For doctor agreement, the numbers in parentheses are the total number of series (out of 49) agreed upon by the assessor. For consistency between doctor and technician or doctor and DL, the numbers in parentheses indicate the number of rescans/recalls/consistencies. Here, rescanning represents a series referred to as good by the physician and bad by another person (technician or DL). Recalls represent a series called bad by the doctor and good by another assessor. The sum of the rescan, recall and number of reconciles is always equal to 49. Here, D1-D5 and T1-T4 represent the same individuals as in Table 2, and D0 is the initial doctor, whose rating was used for deep learning algorithm training. The plus star values in the first six rows indicate that consistency between readers is not a contingent situation; the lower row of stars indicates a higher kappa and series number with agreement being statistically higher than corresponding to agreement between different radiologists (p < 0.05).

with respect to the table, certain observations may be made or repeated. First, there is a large divergence between physicians in this sample, which is rich in datasets with some degree of artifacts. On average, radiologists agreed only on the 34/49 rating, with an average Cohen's kappa of only 0.41. At a minimum, D1 and D2 agreed only on the 25/49 rating. At most, two doctors agreed on 44/49 examinations (D0 and D4). Second, the consistency between radiologists and MRI technicians in assessing diagnostic utility is statistically significant. Third, for physicians whose ratings were used to train the deep learning algorithm (D0), a threshold of 0.5 resulted in the best classification performance, as expected. If a single threshold is chosen for all the rated physicians, the same P (good) ═ 0.5 generally works best.

Fourth, the personalized threshold optimizes the consistency between the deep learning classification algorithm and each radiologist (shaded cells in the table of fig. 5). For example, for more relaxed physicians (D1 and D3, MS lines of the table in table 2), a low threshold P (good) of 0.1 for defining images of sufficient quality works best; for others, a high threshold of 0.5-0.8 will maximize the agreement between the deep learning algorithm and its ranking. With personalized thresholds, the consistency between radiologists and deep learning ratings is statistically better than the consistency between radiologists (bottom row). The use of such an automatic rating program simulates the reader deciding on his own to rescan, thus making the technician different. Such personalized thresholds may be implemented in practice. For example, once such an automated rating algorithm is deployed in a hospital, each radiologist of the hospital may be required to rate a single batch of data sets. The consistency between the deep learning algorithm and the individual doctor will be determined for a range of thresholds. The threshold that optimizes deep learning-individual radiologists consistency, defining the artifact tolerance of a given physician to a given scan indication, may then be saved as a reference for all future exams that particular physician will review.

turning to the table shown in fig. 6, the same deep learning network was used to classify the same 49 series without further training. However, in this case, it is assumed that the patient is scanned to exclude stroke. While in the current implementation the probability output by the deep learning algorithm is independent of the scan indication, it is believed that adjusting the threshold of the imaging volume to make it qualify as good can compensate for the generally lower diagnostic utility of the image required for scans performed to exclude stroke. Since some readers have reached their optimal threshold when the MS scan indicates P (good) {0.5, 0.1, 5e-4, and 1e-6}, the lower threshold is explored. The table of fig. 6 summarizes the results of this survey. Doctor D0, whose initial rating was used to train the deep learning algorithm, does not exist now because no rating was obtained for the reader as "stroke". The asterisk values in the first five rows indicate that consistency between readers is not a casual situation.

With respect to the table, certain observations may be made or repeated. First, a higher agreement is generally reached between physicians on the amount of imaging required to rule out stroke (kappa increases from 0.41 for MS scan indications to 0.5 for stroke). Second, while the consistency between the MRI technician and the radiologist is still statistically equivalent (on average) to that between radiologists, it is observed that one technician (T2) will cause significantly more rescanning than is needed. Third, if the same threshold t of 0.5 is used to separate sufficient diagnostic utility images from insufficient diagnostic utility images for stroke indication, poor agreement will be obtained between deep learning and the doctor's diagnostic utility rating (Cohen's kappa of 0.25). A single optimal threshold P (good) of 1e-6, consistent with the lower diagnostic utility required for images used to exclude stroke, would better separate volumes (Cohen's kappa of 0.47). Fourth, as before, personalization according to physician threshold will optimize volume stratification according to the needs of the individual physician. The more lenient physicians indicated in stroke (stroke D1 and D5 in table 2) required the lowest threshold setting in the deep learning output to maximize consistency, while the higher threshold (P (good) ═ 5e-4) would lead to better consistency for the more stringent physicians (D2, D3, D4). Fifth, with personalized thresholds, the agreement between the deep learning algorithm and the physician and MRI technician, although high, does not exceed statistical significance. However, using such an automated algorithm to make a diagnostic utility determination would prevent a technician such as T2 from scanning fifteen more series than any reading physician would require.

The above findings in the MS background are further summarized in table 3 below. Specifically, the Cohen kappa scores (to determine agreement between different raters) and the serial averages agreed upon by the readers are listed in Table 3.

TABLE 3

Note that a statistically higher consistency is obtained when using the automatic DL algorithm (from the bottom row 3), while adapting the threshold to the needs of each physician (i.e. personalized threshold), rather than simply giving the image to a different reading radiologist. The individualized threshold in this example is equivalent to assigning an individualized P (good) threshold to each physician. For example, doctors D1 and D3 who are less demanding in table 2 are assigned a threshold P (good) of 0.1, and doctor D3 who is more demanding is assigned a threshold P (good) of 0.8. Once the algorithm is deployed in a particular hospital, the radiologist-specific thresholds need to be calibrated once: each physician will be given a number of series, similar to the 49 series data set used for validation here. Based on the physician's rating of these series, he or she will be assigned a threshold that ensures that the best number of series is repeated for that particular physician.

Note that a single doctor, single indication rating may not be sufficient for training a machine learning algorithm that aims to score images for different scan indications. Adapting the threshold to the artifact tolerance level of different radiologists can generate an output tailored to each physician. However, single-condition training may not be sufficient for multiple-condition assessment.

In this regard, an initial algorithm may be trained for multiple scan indications. This may be done by entering the ratings as a specific channel, by having a specific network trained for one scanning indication, or by having a single network trained for multiple indications. For practical implementations, all clinical indications for a scan may be separated in a 3-4 diagnostic utility rating. For example: (1) indications requiring minimal diagnostic utility may be stroke and bleeding; (2) the intermediate indication may comprise an MS; and (3) an indication that the highest indication is required may include screening for epilepsy or brain metastases. In this approach, all incoming scan instructions would be classified into these specific categories and then sent to a network trained specifically for the given category.

In view of the foregoing, determining whether the diagnostic utility of one or more images is sufficient is a function of reading physician and scan instructions (e.g., for diagnostic purposes). Each reading doctor may be given a series of reviews before the algorithm is implemented in the hospital. Based on his or her rating of them, the algorithm may determine a threshold that will separate enough diagnostic utility images from insufficient diagnostic utility images for each doctor and each scan indication. The same threshold will then be maintained for all subsequent scans, which will be ranked by the same physician for the same scan indication. Alternatively, continuous learning may be implemented: for each scan to be reviewed, the algorithm will perform a rating and the physician will have the right to agree or disagree with the rating. Additional incoming streams of data sets for which the physician evaluates their diagnostic utility will then be used to further train and refine the classification algorithm to improve classification performance.

This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.