阅读说明：本技术 将诊断测试集成到实时治疗中的通用设备和方法 (Universal apparatus and method for integrating diagnostic testing into real-time therapy ) 是由 贝杜恩·阿拉亚 古普塔·维卡什 于 2018-03-03 设计创作，主要内容包括：一种用于集成诊断测试和实时治疗的方法和系统,包括用于捕获多个源图像的医学数据采集设备,其中,至少一个源图像包含基准标记。该方法和系统包括低延迟编码器以将捕获的源图像编码为数据流,并且还包括用于捕获传感器数据的环境传感器装置。处理器用于基于捕获的传感器数据和基准标记来基于环境修改源图像,并且传输装置用于将基于环境修改的源图像传输到显示装置。(A method and system for integrated diagnostic testing and real-time therapy includes a medical data acquisition device for capturing a plurality of source images, wherein at least one of the source images contains fiducial markers. The method and system include a low delay encoder to encode a captured source image into a data stream, and further include an environmental sensor device for capturing sensor data. The processor is for modifying the source image based on the environment based on the captured sensor data and the fiducial marker, and the transmitting means is for transmitting the source image modified based on the environment to the display means.)

1. A system for integrating diagnostic testing and real-time therapy, comprising:

a medical data acquisition device configured to capture a plurality of source images, wherein at least one of the source images contains a fiducial marker;

a low-delay encoder configured to encode a plurality of captured source images into a data stream;

an environmental sensor device configured to capture sensor data;

a processor configured to modify a source image based on an environment based on the captured sensor data and the fiducial marker; and

a transmitting device configured to transmit the source image modified based on the environment to the display device.

2. The system of claim 1, wherein the display device is wearable.

3. The system of claim 1, wherein the medical data acquisition device is a medical tool.

4. The system of claim 3, wherein the medical tool comprises an imaging device configured to provide non-invasive images from within a patient's body.

5. The system of claim 1, wherein the processor further modifies the source image based on input from a wearable device.

6. The system of claim 1, wherein the context-based modification of the source image is performed in real-time.

7. The system of claim 1, wherein the environmentally modified based source image comprises an overlay on a patient.

8. A method for integrating diagnostic testing and real-time therapy, comprising:

capturing a plurality of medical source images, wherein at least one of the medical source images contains a fiducial marker;

encoding the captured plurality of medical source images into a data stream using a low delay encoder;

capturing environmental sensor data;

modifying at least one of the medical source images based on an environment based on the captured environmental sensor data and the fiducial marker; and

transmitting at least one of the environmentally modified medical source images to a display device.

9. The method of claim 8, wherein the display device is wearable.

10. The method of claim 8, wherein capturing the plurality of medical source images is performed using a medical tool.

11. The method of claim 10, wherein the medical tool provides non-invasive images from within the patient's body.

12. The method of claim 8, further comprising modifying at least one of the medical source images based on input from a wearable device.

13. The method of claim 8, wherein the context-based modification of at least one of the medical source images is performed in real-time.

14. The method of claim 8 wherein the context-based modification of at least one of the medical source images comprises an overlay on the patient.

15. A non-transitory computer readable medium having instructions stored thereon for integrating diagnostic testing and real-time therapy, the testing and therapy comprising:

capturing a plurality of medical source images, wherein at least one of the plurality of medical source images contains a fiducial marker;

encoding the captured plurality of medical source images into a data stream using a low delay encoder;

capturing environmental sensor data;

modifying at least one of the medical source images based on an environment based on the captured environmental sensor data and the fiducial marker; and

transmitting at least one of the environmentally modified medical source images to a display device.

16. The non-transitory computer-readable medium of claim 15, wherein the display device is wearable.

17. The non-transitory computer readable medium of claim 15, wherein the capturing of the plurality of medical source images is performed with a medical tool that provides non-invasive images from within a patient's body.

18. The non-transitory computer-readable medium of claim 15, further comprising modifying at least one of the medical source images based on input from a wearable device.

19. The non-transitory computer-readable medium of claim 15, wherein the environment-based modification of at least one of the medical source images is performed in real-time.

20. The non-transitory computer-readable medium of claim 15, wherein the environment-based modification of at least one of the medical source images includes an overlay on the patient.

Technical Field

Embodiments of the present invention relate generally to improvements in medical diagnostic and treatment related techniques, and more particularly, to medical imaging devices.

Background

In assisting a patient, medical professionals rely on various devices to help diagnose and treat various ailments and diseases affecting the patient. Medical devices in modern healthcare environments include devices that allow measurement of biological parameters that can be interpreted by a healthcare provider to guide patient care. Such biological parameters may include vital signs such as blood pressure and heart rate, images of the body and its structures, and various other biological parameters, which may be stored on a centralized database called an Electronic Medical Record (EMR).

Typically, a patient seeking medical advice may first undergo a diagnostic test to collect information relating to various biological parameters or generate an image of the patient, which is then passed to a nurse, technician, or physician for review prior to any treatment. The images generated during the diagnostic test are used to help the medical professional decide on a course of treatment that includes a specific procedure that helps to alleviate or repair any pain or injury that afflicts the patient. Examples of diagnostic tests may include radiology, nuclear medicine, ultrasound, and other image generation techniques. In addition, the biological parameters and images may be provided simultaneously with the services provided during treatment via a directly connected display monitor and physical printout.

Disclosure of Invention

Accordingly, embodiments of the present invention are directed to a system and method for integrating diagnostic testing into real-time therapy (i.e., real-time therapy) that substantially obviate one or more problems due to limitations and disadvantages of the related art.

Systems and methods are provided to help provide a user, such as a medical professional, with an environmentally appropriate display of data in real time. The environmentally appropriate data display may be presented as biological information and diagnostic and other images. Data is collected and processed such that environmental information is taken into account when presenting the data to a medical professional such that the display of the data is tailored to the particular setting in which the data is being collected.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, integrated diagnostic testing and real-time therapy (i.e., real-time therapy) includes each method, system, apparatus and computer readable medium configured to visualize data in real-time, including collecting data, transmitting the data to a data transformation device, sensing a first environment in which the data is collected, sensing a second environment in which the data is to be displayed, converting the data based on the environment based on at least the second environment, transmitting environment modified (contextualy modified) data to a data display device, and displaying the environment modified data.

Further, additional advantages may be provided by digitally mapping the environment and tracking area while displaying the environment-modified-based data in an anatomically accurate overlay that is implemented to be generated by fiducial and other trackable markers that may be in a sterile operating and surgical environment.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

In the drawings:

FIG. 1 shows a flow diagram for interfacing with a universal streaming device (universal streaming device) according to an example embodiment;

FIG. 2 shows a flow diagram of components of a device according to an example embodiment;

FIG. 3 is a diagram of an example captured image frame with various data points in accordance with an example embodiment;

FIG. 4 is a diagram of sensor data input and modification of captured data, according to an example embodiment;

FIG. 5 is an illustration of environmental tracking using fiducial markers according to an example embodiment;

FIG. 6 is an illustration of a practitioner using an embodiment of a system according to an example embodiment;

FIG. 7 is a diagram of a magnetic based self-aligning fiducial mark system in accordance with an example embodiment; and

FIG. 8 is an illustration of a practitioner using an embodiment of a system with self-aligning fiducial marks according to an example embodiment.

Detailed Description

Embodiments of the present invention generally relate to a system and method that enables a device to be connected to any medical data collection device or imaging device (analog or digital) and allows near real-time encoding using low-delay encoders and streams of captured data and images (e.g., data streams of images). The medical data collection device may be any type of medical instrument or medical tool or device. Embodiments may include a device connected to any video output port or another connection port of a medical imaging device. Embodiments may include coding and programming to provide environmentally appropriate images from a medical imaging device to a medical professional in real time, preferably during a medical procedure.

The invention may implement an apparatus that is connectable to standard data capture devices used in modern medicine via a standard port, wireless connection, or optical sensor. The apparatus may have the capability to capture relevant data from such devices and wirelessly transmit this modified or unmodified data through various protocols including, but not limited to, Wi-Fi standard, bluetooth, 4G or 5G, during low latency. Low latency means less than 250ms, the latency is measured between data capture and display on the target display device. For example, embodiments are generally directed to systems and methods that may wirelessly transmit data (which may be images) over a closed local network (e.g., Wi-Fi, bluetooth, etc.) or remotely. The device is designed to be generic for at least various medical imaging devices and will allow interpretation of images within and away from the immediate area of the imaging device used.

In addition, the device is capable of streaming images to wearable displays, such as Google Glass or Microsoft Hololens, allowing for the streaming of any medical images or other data without the need for a conventional display.

The apparatus includes the capability to process data using a variety of microprocessors, including a CPU (central processing unit), a GPU (graphics processing unit), and/or a custom ASIC (application specific integrated circuit). The processing power allows for low-latency hardware-based video encoding (i.e., low-latency encoders), file transcoding, encryption encoding, and three-dimensional modification. The apparatus also includes a wireless access point whereby data may be transferred over a secure connection such as WPA2, WPA, or other encoding algorithms.

Further, the apparatus may be configured to act as a relay to control the imaging devices connected to it. The apparatus may be configured to respond to, for example, voice and other commands provided to the wearable display, and an Application Program Interface (API) runs on the general purpose apparatus, where the API controls data collection or imaging settings on an imaging device to which the apparatus is connected. In other words, the device or system may include an updatable command library (library), wherein commands in the command library may be transmitted to the medical device (i.e., diagnostic tool or imaging device) from which data is captured. These commands may be configured to allow wireless control of the connected device through the available APIs.

Another embodiment of the system may have the ability to receive commands and/or other data points from the wearable device or various other sensor systems, which may be utilized to control the connected device through the API and modify the captured data prior to transmission to the wearable unit. This may allow, for example, for three-dimensional manipulation of captured imaging data, such as ultrasound or Computed Tomography (CT) images, which may be displayed in an anatomically correct position directly on the patient during image acquisition, giving the impression of "entering" the patient, e.g., a non-invasive image (non-invasive image). The system may also allow for the superposition of real-time viewing of a patient's physiological or image data during consultation, treatment, or surgery, providing context-based modification of source images (also referred to as medical source images) performed in real-time.

Fiducial markers are objects placed in the field of view of the imaging system that appear in the images produced. The marker may be used as a reference point or metric. In an embodiment, fiducial markers may be used to track specific areas on the patient, allowing for context-based modification of source device data based on the correlation of the fiducial markers with external sensor data, such as optical sensors.

In an embodiment, a patient undergoing a diagnostic imaging scan may receive a scan using fiducial markers in situ, and thus these fiducial markers may be represented in the imaging data. In embodiments, these markings may also be detected by an external sensor and tracked using one or more methods, including optical, laser, infrared, magnetic, or other properties. The external sensor data and the source device data may then be correlated with each other using an image modification algorithm executed by the program, thereby allowing for an environment-based modification of the source image. Additionally, environmental sensor data captured by the environmental sensor device may track the fiducial marker in real-time, allowing real-time, environment-based modification of the source image based on sensor data captured from the environmental sensor device (e.g., captured environmental sensor data).

In embodiments, fiducial markers may be inherent features of the scanned object, such as contours or landmarks, unique covering textures such as clothing or drapes, unique shapes or patterns (such as optical patterns), light sources, magnetic field sources, or other detectable objects. In this way, the fiducial markers may be passive or active in structure and/or function. In an embodiment, the fiducial marker may represent a single trackable point, in another embodiment, the fiducial marker may express one or more vectors to allow it to be tracked in multiple dimensions. Additionally, in embodiments, the fiducial marker may represent an initial marker source, whereby the environmental sensor device may map (map) adjacent environmental features, structures, and textures to extend the features, size, and accuracy of the tracked fiducial marker.

In the context of image-guided medical procedures, fiducial markers may need to be included in the source data prior to field sterilization, but are also visible after field sterilization and accessible by environmental tracking equipment. Additionally, fiducial markers may need to be overlaid on the sterile surgical field/drape and remain in a fixed position relative to the subject when tracked. In this manner, the fiducial marker may need to be present both before and after sterilization of the surgical field, including in the sterile field without compromising sterility, and without changing its position before and after the surgical field sterilization procedure.

In embodiments, self-aligning and complementarily attracting fiducial markers can be used to locate these basic components of precise environmental tracking during the image-guided procedure. In such embodiments, adhesive or non-adhesive based fiducial markers may be fixed to a particular part of the imaging subject during the initial data acquisition. The fiducial markers may remain in an unaltered position during site sterilization and be covered at the time of placement of the sterile drape. Subsequently, the complementary sterile fiducial marker may be attached to the initial marker by the drape using an attractive force (e.g., by a magnetic or electronic force of attraction). The two fiducial markers may self-align based on their basic characteristics when in proximity to each other and the fiducial markers on the drape may be tracked by the environmental sensor equipment, serving as an accurate substitute for the non-sterile markers. In an embodiment, surrounding drape features and textures may also be drawn to expand the trackable region. In other embodiments, multiple fiducial markers may be used, non-sterile markers may be sterilized, environmental sensor devices may be able to track covered markers, or marker positions may be accurately defined after sterilization.

In another embodiment, additional trackable markers may be placed to extend the trackable region. The relationship of these additional markers to the initially placed markers can be inferred to extend the trackable field and/or provide additional reference points for environment-based image modification. In this manner, additional trackable points may be used if the initially placed fiducial marker cannot be adequately tracked by the environmental sensor.

From the data capture device, the capture data encoding may be hardware accelerated to minimize latency. Low latency is important to provide real-time interpretation of images (e.g., source images) acquired by a data source device. The apparatus is robustly designed so that it can be adapted for use in any ultra-low delay streaming application, particularly over wireless networks.