阅读说明：本技术 在检查或介入程序中的增强现实用户引导 (Augmented reality user guidance in examination or interventional procedures ) 是由 M·J·H·登哈尔托赫 J·奥利万贝斯科斯 T·埃伦巴斯 W·E·P·范德斯特伦 D·S·A 于 2019-09-05 设计创作，主要内容包括：本发明涉及检查或介入程序中的指导。为了便于在诸如手术室或导管室的医学环境中的信息提供,提供了一种用于医学装备的增强现实显示设备(10),其包括数据输入单元(12)、处理单元(20)和显示单元(22)。所述数据输入单元被配置为接收至少一个医疗器械的显示操作参数(14)。所述数据输入单元还被配置为接收至少一个医疗器械相对于所述显示单元的相对定位信息(16)和所述用户的观看方向信息(18)。所述处理单元被配置为基于所述相对定位信息和所述观看方向信息来检测所述医疗器械中的至少一个是否在所述用户的视场中。所述处理单元还被配置为基于检测到的医疗器械的操作参数来生成显示数据。所述显示单元被配置为：如果所述医疗器械在用户的视场内,则基于所述显示数据来投影与对用户可见的所述医疗器械的所述操作参数有关的信息(24)。所述显示单元被配置为将所述信息显示为叠加到现实的可见表示。(The present invention relates to guidance in examination or interventional procedures. To facilitate information provision in a medical environment such as an operating room or a catheter room, an augmented reality display device (10) for medical equipment is provided, comprising a data input unit (12), a processing unit (20) and a display unit (22). The data input unit is configured to receive display operating parameters (14) of at least one medical instrument. The data input unit is further configured to receive relative positioning information (16) of at least one medical instrument with respect to the display unit and viewing direction information (18) of the user. The processing unit is configured to detect whether at least one of the medical instruments is in a field of view of the user based on the relative positioning information and the viewing direction information. The processing unit is further configured to generate display data based on the detected operating parameter of the medical instrument. The display unit is configured to: projecting information (24) relating to the operating parameter of the medical instrument visible to a user based on the display data if the medical instrument is within a field of view of the user. The display unit is configured to display the information as a visual representation superimposed to reality.)

1. An augmented reality display device (10) for medical equipment, comprising:

a data input unit (12);

a processing unit (20); and

a display unit (22);

wherein the data input unit is configured to: receiving a displayed operating parameter (14) of at least one medical instrument; and receiving relative positioning information (16) of at least one medical instrument with respect to the display unit and viewing direction information (18) of the user;

wherein the processing unit is configured to: detecting whether at least one of the medical instruments is in a field of view of the user based on the relative positioning information and the viewing direction information; and identifying at least one of the displayed operating parameters of the detected medical instrument; and generating display data comprising information (24) indicative of at least one of the identified operating parameters of the medical instrument in the user's field of view, and

wherein the display unit is configured to project the generated display data as a visual representation superimposed to reality.

2. The device of claim 1, wherein the generated information includes replicated display data that replicates at least one of the identified operating parameters of the detected medical instrument.

3. The device of claim 1 or 2, wherein the data input unit is configured to receive the relative positioning information:

-localization information (26) of the at least one medical instrument;

position information (28) of the display unit; and

orientation information (30) of the user;

wherein the processing unit is configured to derive the relative positioning information with respect to the user from this information.

4. The device of claim 1, 2 or 3, wherein the visual representation is displayed as an overlay to reality depending on the presence of the medical instrument in the field of view of the user; and is

Wherein the display unit is configured to project the visible representation into the vicinity of the medical instrument within the field of view, comprising at least one from the group of: i) adjacent to the medical instrument and ii) superimposed to the medical instrument; and is

Wherein the display unit is further configured to project a device indicator for pointing to or highlighting the at least one medical instrument for which the visual representation is presented.

5. The device according to any one of the preceding claims, wherein the data input unit is configured to receive user settings (32) comprising at least information about an assignment to at least one of a plurality of predetermined user categories (34); and is

Wherein the processing unit is configured to adjust the display data based on the user setting.

6. The device of any one of the preceding claims, wherein the display unit is configured to present the visual representation of the operating parameter of the medical instrument in a graphical setting based on presentation of the respective operating parameter on a display of the medical instrument; and is

Wherein the visual representation is a magnified reproduction of at least a portion of the presentation on the medical instrument.

7. The device according to any one of the preceding claims, wherein the display unit is a head mounted display (38); and is

Wherein the head mounted display comprises a display device (40) configured to: i) allowing the user to browse through at least a portion of the user's field of view, and ii) providing the visual representation on a display element within the user's field of view.

8. The device according to any one of the preceding claims, wherein a viewing direction identifier (42) is provided, the viewing direction identifier being configured to detect a viewing direction of the user relative to at least one medical instrument and to provide the detected viewing direction (44) to the data input unit; and/or

Wherein a medical instrument identifier (46) is provided, the medical instrument identifier being configured to identify at least one medical instrument and to provide the display operating parameters (48) of the identified at least one medical instrument to the data input unit.

9. The device of any preceding claim, wherein the processing unit is configured to temporarily modify the visible copy when at least one of the display operating parameters reaches a predetermined threshold; and wherein the display unit is configured to project the modified visible copy to the user as an interactive signal.

10. An information system (50) for medical equipment, the system comprising:

at least one augmented reality display device (52) according to one of the preceding claims;

at least one medical instrument (54) having display operating parameters; and

a display operation parameter transmitting device (56);

wherein the display operating parameter transmitting means is configured to provide information regarding display operating parameters from the at least one medical instrument to the data input unit of the at least one augmented reality display device.

11. The system of claim 10, wherein the display operating parameter transmitting means further comprises:

i) a data connection link between the at least one medical instrument and the data input unit of the at least one augmented reality display device; and/or

ii) a camera system providing optical coverage of a display area on the at least one medical instrument.

12. The system according to claim 10 or 11, wherein a position tracking device (58) is provided for determining a current position of the at least one medical instrument relative to the object using one of the at least one augmented reality display device; and is

Wherein the positioning and tracking device comprises at least one from the group of: optical cameras, depth sensors, and electromagnetic and optical tracking with markers or tags.

13. A method (100) for providing operating parameters of a medical instrument, the method comprising the steps of:

a) receiving (102) displayed operating parameters of at least one medical instrument;

b) receiving (104) relative positioning information of at least one medical instrument in relation to a display unit of an augmented reality display device for medical equipment and viewing direction information of the user;

c1) detecting (106) whether at least one of the medical instruments is in the user's field of view based on the relative positioning information and the viewing direction information, and

c2) identifying at least one of the displayed operating parameters of the detected medical instrument;

d) generating (108) display data comprising information indicative of at least one of the identified operating parameters of the detected medical instrument; and is

e) The generated display data is projected (110) into a visual representation, which is superimposed as augmented reality to the live image stream.

14. A computer program element for controlling an apparatus according to one of claims 1 to 12, which, when being executed by a processing unit, is adapted to perform the method steps of claim 13.

15. A computer readable medium having stored the program element of claim 14.

Technical Field

The present invention relates to guidance in examination or interventional procedures, and in particular to an augmented reality display device, a method of providing operating parameters of a medical instrument for an information system of a medical equipment, a computer program element and a computer readable medium.

Background

In medical examination or interventional procedures, for example in a catheter laboratory (cathlab) or an interventional or operating room of a hospital, various different equipment is used. For example, the equipment may belong to the group of treatment devices, diagnostic devices, imaging devices, and other support devices. For example, a wide variety of data is provided to clinical personnel such as surgeons, technical equipment operators, nurses, and the like. Central and auxiliary displays are provided to display information before, during and after an examination or intervention. A part of the information is displayed on a display element of the device itself, like the flowing stream of injected contrast agent. Furthermore, in order to provide additional information, so-called augmented reality is also used. For example, WO 2018052966 a1 describes augmented reality surgical technique guidance. In one example, non-visual information is presented to the user through augmented reality (e.g., information about the interior of an object). However, it has been shown that sometimes it is cumbersome for a user to identify or view the desired information.

Disclosure of Invention

Thus, there may be a need to facilitate information provision in a medical environment such as an operating room or a catheter lab.

The object of the invention is solved by the subject matter of the independent claims; further embodiments are incorporated in the dependent claims. It should be noted that the following described aspects of the invention also apply to the augmented reality display device, the information system for the medical instrument and the method for providing operating parameters of the medical instrument.

According to the present invention, an augmented reality display device for medical equipment is provided. The apparatus comprises: a data input unit, a processing unit, and a display unit. The data input unit is configured to receive display operating parameters of the at least one medical instrument and to receive information of a relative positioning of the at least one medical instrument with respect to the display unit and information of a viewing direction of the user. The processing unit is configured to detect whether at least one of the medical instruments is in a field of view of the user based on the relative positioning information and the viewing direction information; and identifying at least one display operating parameter of the detected medical instrument; and generating display data including information indicative of at least one identified operating parameter of the medical instrument within the field of view of the user. Furthermore, the display unit is configured to project the generated display data as a visual representation superimposed to reality.

According to an example, the generated information includes replicated display data that replicates at least one of the identified operating parameters of the detected medical instrument. In other words, the processing unit is configured to generate replicated display data comprising a replication of at least a portion of the operating parameters of the medical instrument visible to the user. The display unit is configured to display a copy of the operating parameter as a visual representation superimposed to reality.

The visual representation is superimposed on the reality seen by the particular user. Thus, the copy of the operational parameters is only visible to the user.

By providing duplicate information in augmented reality, it can be ensured that all information can be provided in a convenient and targeted manner. Thus, the physician or other user is permanently informed of the settings and status of the device used (particularly the procedure-critical device). The physician may continue to concentrate on his tasks when the device display screen is not visible or difficult to read, and need not ask other personnel (e.g., a technician or nurse) to pass the displayed information to him. The physician also need not physically move from his/her location to better view the device display operating parameters. Thus, continuous monitoring can be performed, if desired. The workflow is thus further optimized. In extreme cases, it is ensured that the correct settings can be checked and used and that the clinician operates according to accurate information. By providing additional information (i.e., duplicate information) only when the device is in the user's field of view, it can be ensured that the field of view is not cluttered with information that is not currently of interest, based on the following assumptions: only information of particular interest pertaining to the devices the user is looking for, i.e. when the user looks in the direction of these devices.

The copy of the operating parameters of the medical instrument is visible only to a particular user using the augmented reality display device. Thus, although the information is provided in a duplicated and thus redundant manner, no separate display is used, which would clutter the space in the room, in particular the space around the object support.

According to an example, for the relative positioning information, the data input unit is configured to receive position information of the at least one medical instrument, position information of the display unit and orientation information of the user. Furthermore, the processing unit is configured to derive relative positioning information about the user from the information.

According to an example, the visual representation is displayed as an overlay to reality, depending on the presence of the medical instrument in the field of view of the user. Further, additionally or alternatively, the display unit is configured to project the visual representation to the vicinity of the medical instrument within a field of view, comprising at least one from the group of: i) in close proximity to the medical instrument and ii) superimposed on the medical instrument. Still further, additionally or alternatively, the display unit is further configured to project a device indicator for pointing to or highlighting the at least one medical instrument for which the visual representation is presented.

According to an example, the data input unit is configured to receive user settings comprising at least information about an assignment to at least one of a plurality of predetermined user categories. Further, the processing unit is configured to adjust the display data based on user settings.

Thus, for example, assigning different users to different user categories, information indicative of different display operating parameters of the medical instrument may be generated. When users are provided with their own augmented reality display devices, adjusted or customized display data may be generated and presented to each user in augmented reality, wherein the information presented to the user may, for example, include only copies of those display operating parameters of particular interest to the user.

According to one example, the display unit is a head mounted display comprising a display device configured to: i) allowing a user to view at least a portion of the user's field of view, and ii) providing a visual representation on a display element within the user's field of view.

According to an example, the processing unit is configured to temporarily modify the visible copy when at least one of the display operating parameters reaches a predetermined threshold. Further, the display unit is configured to project the modified visible copy to a user as an interactive signal.

Thus, an intuitive alarm type signal is provided.

According to the invention, an information system for a medical instrument is also provided. The system comprises at least one augmented reality display device according to one of the above described examples. The system also includes at least one medical instrument having a display operating parameter and a display operating parameter transmitting device. The display operating parameter transmitting means is configured to provide information regarding the display operating parameter from the at least one medical instrument to the data input unit of the at least one augmented reality display device.

According to the present invention, there is also provided a method for providing operating parameters of medical equipment. The method comprises the following steps:

a) receiving a display operating parameter of at least one medical instrument;

b) receiving relative positioning information of at least one medical instrument related to a display unit of an augmented reality display device for medical equipment and viewing direction information of the user;

c) detecting whether at least one of the medical instruments is in a field of view of a user based on the relative positioning information and the viewing direction information, and identifying at least one of the detected display operating parameters of the medical instrument;

d) generating display data comprising information indicative of at least one of the identified operating parameters of the detected medical instrument; and is

e) The generated display data is projected as a visual representation which is superimposed as augmented reality into the reality of the real-time image stream.

According to one example, a copy of displayed information is provided to a user with an augmented reality device, such as a head mounted display. By providing a display unit, the settings of the device may be identified using an Application Programming Interface (API), screenshots or any other technique, and information may be displayed above or beside the device. The location of the device itself may be discovered using, for example, RFID tags, depth camera tracking, optical camera tracking, or other tracking and positioning systems. The tracking device may be integrated with the augmented reality device or may be a separate device. Since the content provided on the display unit is artificially presented, other functions may be provided in addition to the copied display content. In one example, the copied information is provided such that the information is always located on top of the respective device and is also facing the viewer. Providing the replicated display information may also be referred to as presenting the remote device display information in augmented reality.

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.

Drawings

Exemplary embodiments of the invention will be described below with reference to the accompanying drawings:

fig. 1 shows a schematic diagram of an augmented reality display device for medical equipment.

Fig. 2 illustrates another example of an augmented reality display device.

Fig. 3 illustrates another example of an augmented reality display device, wherein the display is provided as a head mounted display.

Fig. 4 shows yet another example of an augmented reality display device.

Fig. 5 shows an information system for medical equipment with an example of an augmented reality display device.

Fig. 6 shows basic steps of an example of a method for providing operating parameters of a medical equipment.

Detailed Description

Fig. 1 shows a schematic diagram of an augmented reality display device 10 for medical equipment. The augmented reality display device 10 includes a data input unit 12. The data input unit 12 is configured to receive displayed operating parameters 14 of at least one medical instrument. The data input unit 12 is further configured to receive relative positioning information 16 of at least one medical instrument with respect to the display unit and viewing direction information 18 of the user. The augmented reality display device 10 further comprises a processing unit 20. The processing unit 20 is configured to detect whether at least one of the medical instruments is in the field of view of the user based on the relative positioning information and the viewing direction information. The processing unit 20 is further configured to identify at least one of the identified operating parameters of the detected medical instrument and to generate display data comprising information indicative of at least one of the identified operating parameters of the detected medical instrument. The augmented reality display device 10 also includes a display unit 22. The display unit 22 is further configured to project the generated display data as a visual representation superimposed to reality.

Thus, the display unit 22 is configured to project information 24 indicative of at least a part of the display operating parameters of the medical instrument in the field of view of the user in augmented reality, for example as an overlay to a real-time image stream.

In an example, the generated information includes replicated display data that replicates at least one of the identified operating parameters of the detected medical instrument. Thus, the processing unit 20 may be configured to generate replicated display data that replicates at least one of the detected operating parameters of the medical instrument, and the display unit 22 is configured to project said replication of the operating parameters as a visual representation superimposed to reality.

The term "medical equipment" relates to an instrument for medical purposes, e.g. clinical purposes. For example, the term relates to an instrument for use in an examination or interventional procedure of a subject.

The term "medical instrument" especially relates to catheter laboratory instruments and operating room instruments, such as object monitoring devices, treatment devices, diagnostic devices, imaging devices and other support devices. One example of a treatment or support device is a power injector, which is used to apply a controlled dose of a selected substance, for example, during an X-ray imaging procedure. The injections may be provided, for example, at intervals or in a constant flow. Other examples of medical instruments are intravascular ultrasound (IVUS) carts or consoles, or Fractional Flow Reserve (FFR) measurement carts or consoles, or Optical Coherence Tomography (OCT) measurement carts or consoles.

The term "displaying an operating parameter" relates to a parameter indicated or displayed on the medical instrument, for example during use of the medical instrument. The parameter is thus related to the operation of the medical instrument. The parameter may be a value required to control or adjust the operation of the medical instrument. The parameter may also be a detected value related to the object, which value may be a measurement made during operation of the medical instrument.

In an example, values relating to device settings or respective states are provided before, during and after use.

The term "relative positioning information" relates to information about the current position or arrangement of a medical instrument relative to a user of the augmented reality display device.

The term "viewing direction information" relates to the current direction of the user's view for determining which part of the surroundings the user is actually viewing. As an example, a marker attached to the user is provided, for example in the form of glasses equipped with such a marker, which allows detecting the viewing direction.

The term "field of view" relates to the area actually covered by the user's vision. The field of view may be provided as a predetermined angle based on the line of sight direction.

The term "copying display data" relates to displaying data that has been provided by the medical instrument. Thus, the data displayed by the augmented reality display device is at least a partial copy of the data provided by the medical instrument. However, the term "replication" relates to replication for a user, as the replicated display data may not be visible to other workers in the room. Of course, if they are also provided with an example of an augmented reality display device, they will also see (the same or other) copied display data.

In one example, replicated display data is provided for devices critical to the outcome of the program. The device may also be referred to as a program critical device.

In an example, the display unit is configured to display a visual representation of reality superimposed to the real-time image stream. The live image stream may be provided by a display device (e.g. a small monitor) or the reality may be viewed directly by the user through a lens which also provides the replicated data.

Since a duplicate display is generated, continuous monitoring is supported. Since the information displayed in the display unit is easy to view, convenient information is provided to the worker. Furthermore, since only device information in the user's field of view is presented, it is convenient for the user to perceive the information. For example, important system settings and other information may be presented and provided above the device or in a closed position. In one option, depending on the user, i.e., the wearer of the augmented reality device, the data may be displayed at a different device, or even at the same device.

Alternatively, fig. 2 shows an example, wherein the data input unit 12 is configured to receive for the relative positioning information 26 of the at least one medical instrument, position information 28 of the display unit and orientation information 30 of the user. Furthermore, the processing unit is configured to derive relative positioning information with respect to the user from the information, i.e. from the positioning information, the position information and the orientation information. Optionally, viewing direction information 18 is also derived from this information. In another alternative, the viewing direction information 18 is additionally provided as separate information.

For example, the display unit is provided in the form of a wearable device (e.g., glasses) or other device carried by the user (e.g., a head-mounted or head-mounted device). If the information also comprises information about the angle of rotation of the display unit with respect to the surroundings, the viewing direction may also be derived from the positioning information,

in an example not shown in further detail, the replicated visual representation is displayed as an overlay to reality, depending on the presence of the medical instrument in the user's field of view.

For example, the replicated visual representation is displayed by the display unit 22 only if the medical instrument is in the field of view of the user. The range covered by the "field of view" may be predetermined and adjusted by the user.

In an example not shown in detail, the display unit 22 is configured to project the visual representation in the field of view in the vicinity of the medical instrument as being adjacent to the medical instrument. Alternatively or additionally, the display unit 22 is configured to project the visual representation in the field of view in the vicinity of the medical instrument as superimposed on the medical instrument. The display unit 22 is further configured to project a device indicator (not shown) for pointing to or highlighting the at least one medical instrument for which the visual representation is presented. In one example, an arrow is provided that points from the visible representation to the respective medical instrument.

In an option, the display unit is provided with the option of projecting a device indicator for pointing at the at least one medical instrument for which the visual representation is presented.

The term "nearby" relates to an arrangement or positioning wherein the user can directly identify to which device in his view the presented information is assigned. In a very crowded environment with many devices, the respective arrangements are closer than for a very compact device setup.

The term "adjacent" relates to an arrangement or positioning, wherein in the field of view of the user the respective information is in close proximity or in direct contact with the visible device to which the information relates.

The term "overlay" relates to an arrangement or positioning in which, in the field of view of a user, information at least partially overlaps with a visual device to which the information relates. For example, in the user's field of view, the information is arranged across the entire device. In an example, the information is arranged within the respective device in the field of view of the user. In another example, in the user's field of view, the information extends to an area outside the respective device.

Fig. 2 also shows another option for an example of the augmented reality display device 10. The data input unit 12 is configured to receive user settings 32, said user settings 32 comprising at least information about the assignment to at least one of a plurality of predetermined user categories 34, for example stored in a data storage 35 connected to the processing unit 20, or stored by the processing unit 20 itself. The user category 34 may also be provided by other external units or devices. The processing unit 20 is configured to adjust the display data based on user settings. In addition to the other options shown, the option of the user setting the entry is provided.

Thus, a nurse may be provided with different types of information displayed on the device than a surgeon. Thus, the visible copy is provided as a lean copy or a condensed copy. For example, the user category provides different respective duplicate display settings for the first and second categories (e.g., "physician" and "staff"). For example, the equipment and maintenance data is presented to an employee, wherein the operational data is provided to a physician, such as a surgeon. For example, in the case of a contrast medium injector, a worker (such as a nurse) is provided with duplicated data of the remaining time until a new contrast medium injector must be placed. The delivered cumulative contrast volume is provided to the physician.

In one example, as an option, a plurality of augmented reality display devices are provided. Further, different user categories with assigned duplicate display settings are provided; and the duplicate display data is adjusted based on the selected category and the respectively assigned duplicate display settings. In one example, different data is displayed for the same device for different user settings. In another example, data is displayed for different devices for different user settings.

In an example not further shown, the display unit 22 is configured to present the visual representation of the operating parameters of the medical instrument in graphical settings based on the presentation of the respective operating parameters on the display of the medical instrument. The visual representation is provided as an enlarged copy of at least a portion of the representation on the medical instrument.

In one example, the size of the rendered copy graphic is independent of distance, i.e., easy to read even if the device is remote from the clinician.

In an option, a display unit is provided that is configured to display a visual representation of the operating parameters of the medical instrument in a graphical setting that is based on the presentation of the individual operating parameters on the display of the medical instrument, but without the magnified features.

In another example, the visualization is static. In yet another example, the visualization is non-static, and the content is altered given some preset threshold (such as an alarm), or interacted with using gesture or voice control. For example, the graphic may be zoomed in when the device is above a certain threshold, and return to its original size when "tapped" using the gestural control, or even closer to the clinician depending on urgency.

Fig. 3 shows another example of an augmented reality display device 10, wherein the display is provided as a head mounted display 38. The head mounted display 38 includes a display device 40, the display device 40 being configured to allow a user to view at least a portion of the user's field of view. The display device 40 is also configured to provide a visual representation on the display element within the field of view of the user. In this example, the head mounted display 38 is worn by a user 39. The head mounted display 38 may include a central portion 41 arranged like glasses that is at least partially within the field of view of the user. The central portion 41 may be provided as a left portion and a right portion for each eye of the user. The projection unit 43 is instructed to arrange the projection on, for example, a lenticular central part 41 to provide a copy of the operating parameters of the medical instrument visible to the user, so that the visible representation is superimposed on the reality seen by the respective user.

In one option, the display unit is a head mounted display, but it is not required that the head mounted display comprises a display device configured to: i) allowing a user to view at least a portion of the user's field of view, and ii) providing a visual representation on a display element within the user's field of view.

In an example, the display device is a transparent element and the visual representation is displayed on the display element, e.g. by a projector element. In another example, the visual representation is displayed in the field of view of the user by holographic techniques. In another example, the head mounted display is a product such as HoloLens (registered trademark) from Microsoft corporation.

Fig. 4 shows yet another example of an augmented reality display device 10. A viewing direction identifier 42 is provided, which is configured to detect a viewing direction of the user relative to the at least one medical instrument and to provide the detected viewing direction 44 to the data input unit 12. Additionally or alternatively, a medical instrument identifier 46 is provided, which is configured to identify at least one medical instrument and to provide the display operating parameters 14 of the identified at least one medical instrument to the data input unit 12.

In one example, the processing unit 20 is configured to temporarily modify the visible copy when at least one of the display operating parameters reaches a predetermined threshold. The display unit 22 is configured to project the modified visible copy to a user as an interactive signal. The modified visible copy thus acts as an interactive signal to the user. A warning signal is provided to the user.

Fig. 5 shows an information system 50 for a medical instrument. The system 50 includes at least one augmented reality display device 52, which is provided as one of the examples of augmented reality display device 10 described above. The system 50 also includes at least one example of a medical instrument 54 having displayed operating parameters. The system 50 further comprises a display operating parameter transmitting means 56, the display operating parameter transmitting means 56 being configured to provide information about the display operating parameter from the at least one medical instrument 54 to the data input unit of the at least one augmented reality display device 52.

The "display operation parameter transmission means" may also be referred to as transmission means for transmitting the display operation parameter. The display operating parameter is provided by, i.e. displayed by, at least one medical instrument. "display operating parameter transmitting means" are provided to feed corresponding information about the displayed content to the processing unit via the data input unit, so that the processing unit can generate duplicated display data based on the transmitted or forwarded information about what content is displayed on the device.

In one example (shown as an option), the display operating parameter transmitting means 56 comprises a data connection link 55 between the at least one medical instrument 54 and the data input unit of the at least one augmented reality display device, said data connection link being indicated by two dashed arrows. Additionally or alternatively, it is also provided as an option that the display operating parameter transmitting means 56 further comprise a camera system 57, said camera system 57 providing an optical overlay on the display area on the at least one medical instrument 54, for example connected to the data input unit via a data connection link.

For the at least one medical instrument 54, for example, a contrast agent injector is provided. A contrast agent injector may be provided for injecting contrast agent via the connection 53 to an object 51, for example an object for examination. The medical instrument 54 has a display 59, the display 59 indicating for example a flow stream of 2ml/s, a duration of 8s and an available rate of 240mg l/ml.

By way of example, other equipment of a catheter laboratory is shown. The object is arranged on a support table 60. Furthermore, for X-ray imaging, the C-arm arrangement 62 is provided with a movably supported C-arm 64 and an X-ray source 66 and an X-ray detector 68 attached to the ends of the C-arm. A ceiling support 70 having rails and a movable carriage is provided for mounting the C-arm apparatus 92 to the ceiling of a building structure.

The illustrated console or control table 72 includes a user interface tool (e.g., a keyboard) and a number of primary displays 74. An additional auxiliary display 76 is provided which is attached to the patient support 60.

The user 39 is schematically indicated to be in the vicinity of the patient table wearing an augmented reality display device 52 provided as a head mounted display. Depending on the placement of the medical instrument 54, the display 59 may be difficult to recognize by the user 39 (e.g., surgeon). To this end, the augmented reality display device provides a visual representation of the operating parameters of the medical instrument in a replicated form that is only visible to the user. When the medical instrument 54 is in the user's field of view, duplicate information is shown alongside the medical instrument 54. The display unit of the augmented reality display device 52 is configured to display a copy of the operating parameters to be superimposed on the reality augmented reality.

The data connection may be a wired connection or a wireless connection. The data may be obtained directly from the medical instrument or may be collected in the form of data provided to a display of the medical instrument. Data present on the medical instrument is then acquired to generate replicated data for a display unit of the augmented reality display device. In one example, a plurality of medical instruments are connected to a bus system that provides data connectivity.

The camera system is provided as at least one camera providing images, i.e. a sequence of displays on the medical instrument. The image is then captured to extract corresponding information to generate replicated data for a display unit of the augmented reality display device. In an example, a plurality of smaller cameras are provided to cover a display of a plurality of medical instruments.

In one example (also shown as an option), a position tracking device 58 is provided to determine a current position of the at least one medical instrument relative to the object using one of the at least one augmented reality display device; the positioning and tracking device 58 comprises at least one of the group of: optical cameras, depth sensors, and electromagnetic and optical tracking with markers or tags. In a further option, a position tracking device is provided for determining a current position of the at least one medical instrument relative to the object using one of the at least one augmented reality display device, but the position tracking device does not include at least one of the group of an optical camera and electromagnetic tracking with markers or tags.

In an example, the position of the user is tracked to determine the relative position of the instrument from absolute position information.

Fig. 6 shows a method 100 for providing operating parameters of a medical equipment. The method 100 includes the following steps. In a first receiving step 102 (also referred to as step a), displayed operating parameters of at least one medical instrument are received. In a second receiving step 104 (also referred to as step b), relative positioning information of the at least one medical instrument with respect to a display unit of an augmented reality display device for the medical instrument, and viewing direction information of the user are received. The first receiving step and the second receiving step may be performed simultaneously or in the order of the first receiving step and then the second receiving step, or vice versa. In the detecting and identifying step 106 (also referred to as step c), it is detected whether at least one of the medical instruments is in the field of view of the user based on the relative positioning information and the viewing direction information. If so, at least one of the detected display operating parameters of the medical instrument is identified. The detection sub-step is also referred to herein as step c1), and the identification sub-step is also referred to herein as step c 2).

In a generating step 108, also referred to as step d), display data is generated, the display data comprising information indicative of at least one of the identified operating parameters of the detected medical instrument. In a projection step 110 (also referred to as step e), the generated display data is projected as a visual representation superimposed on reality. Thus, the information may be displayed in augmented reality, for example as an overlay of a real-time image stream.

In one example, step d) includes generating duplicate display data that duplicates at least one of the identified operating parameters of the detected medical instrument. And step e) comprises: projecting a visual copy of the operating parameters of the medical instrument for the user based on the copied display data if the medical instrument is within the field of view of the user, wherein the visual copy is displayed as a visual representation superimposed in the real-time image stream reality as augmented reality.

In another exemplary embodiment of the invention, a computer program or a computer program element is provided, which is characterized in that it is adapted to perform the method steps of the method according to one of the preceding embodiments on a suitable system.

The computer program element may thus be stored on a computing unit, which may also be part of an embodiment of the present invention. The calculation unit may be adapted to perform or cause the performance of the steps of the above-described method. Furthermore, it may be adapted to operate the components of the device described above. The computing unit may be adapted to operate automatically and/or to execute commands of a user. The computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the methods of the present invention.

This exemplary embodiment of the invention overlays the computer program that originally used the invention and the computer program that turned an existing program into a program that used the invention by means of an update.

Furthermore, the computer program element may be able to provide all necessary steps to complete the flow of an exemplary embodiment of the method as described above.

According to another exemplary embodiment of the present invention, a computer-readable medium, such as a CD-ROM, is proposed, wherein the computer-readable medium has stored thereon a computer program element, which is described in the previous section. A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.

However, the computer program may also be provided via a network, such as the world wide web, and may be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform one of the previously described embodiments of the present invention.

It has to be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to apparatus type claims. However, a person skilled in the art can derive from the above and the following description that, unless otherwise indicated, in addition to any combination of features belonging to the same type of subject matter also any combination between features relating to different subject matters is considered to be disclosed by the present application. However, all features can be combined, providing a synergistic effect beyond the simple addition of the features.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims shall not be construed as limiting the scope.