System and method for location sensor based branch prediction
阅读说明:本技术 用于基于定位传感器的分支预测的系统和方法 (System and method for location sensor based branch prediction ) 是由 赫德耶·拉菲-塔里 于 2019-05-28 设计创作,主要内容包括:提供了用于基于定位传感器的分支预测的系统和方法。在一个方面,该方法包括:基于由用于器械的一个或更多个定位传感器的集合生成的第一定位数据来确定器械的第一取向,以及基于第二定位数据来确定器械在第二时间的第二取向。器械的远端在第一时间和第二时间位于模型的第一段内并且第一段分支成两个或更多个子段。该方法还包括:确定指示第一取向与第二取向之间的差异的数据,以及基于指示差异的数据来确定器械将前进到子段中的第一子段中的预测。(Systems and methods for location sensor based branch prediction are provided. In one aspect, the method comprises: the method further includes determining a first orientation of the instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, and determining a second orientation of the instrument at a second time based on second positioning data. The distal end of the instrument is located within a first segment of the model at a first time and a second time and the first segment branches into two or more sub-segments. The method further comprises the following steps: the method further includes determining data indicative of a difference between the first orientation and the second orientation, and determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.)
1. A system, comprising:
a processor; and
at least one computer-readable memory in communication with the processor and having stored thereon a model of a luminal network of a patient, the memory further having stored thereon computer-executable instructions for causing the processor to:
determining a first orientation of an instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data indicative of a position of the instrument in a positioning sensor coordinate system at a first time;
determining a second orientation of the instrument at a second time based on second positioning data generated by a set of positioning sensors, a distal end of the instrument being located within a first segment of the model at the first time and the second time and the first segment branching into two or more sub-segments;
determining data indicative of a difference between the first orientation and the second orientation; and
determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
2. The system of claim 1, wherein the prediction comprises data indicative of a probability that the instrument will advance into the first sub-segment.
3. The system of claim 1, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a prediction that the instrument will advance into a second of the sub-segments based on the data indicative of the difference.
4. The system of claim 1, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a third orientation of the instrument at a third time based on third positioning data generated by the set of positioning sensors;
determining an angle between the third orientation and the first orientation;
comparing the angle to a threshold angle; and
in response to the angle being greater than the threshold angle, updating the prediction that the instrument will advance into the first sub-segment.
5. The system of claim 1, wherein:
the set of positioning sensors includes an Electromagnetic (EM) sensor configured to generate data indicative of an orientation of the EM sensor within an EM field,
the EM field is generated by an EM field generator,
the memory also has stored thereon computer-executable instructions for causing the processor to: determining a yaw angle between the first orientation and the second orientation based on data generated by the EM sensor.
6. The system of claim 5, wherein a yaw axis of the instrument determined based on the EM field is substantially aligned with an orientation difference between a first and second of the subsections.
7. The system of claim 1, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
8. The system of claim 1, wherein the positioning data is not registered to a model coordinate system of the model.
9. The system of claim 1, wherein the memory further has stored thereon:
a target path to a target within the model,
a contralateral registration path comprising: driving the instrument along a first branch of the luminal network outside the target path, returning the instrument to the target path, and driving the instrument along a second branch that is part of the target path, an
Computer-executable instructions for causing the processor to: determining whether the instrument is positioned along the contralateral registration path based on a prediction that the instrument will advance into the first sub-segment.
10. The system of claim 9, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path; and
providing an indication that contralateral registration is unsuccessful in response to determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path.
11. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:
determining a first orientation of an instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data indicative of a position of the instrument in a positioning sensor coordinate system at a first time;
determining a second orientation of the instrument at a second time based on second positioning data generated by a set of positioning sensors, a distal end of the instrument being located within a first segment of a model at the first and second times and the first segment branching into two or more sub-segments, the model being stored in a memory and modeling a luminal network of a patient;
determining data indicative of a difference between the first orientation and the second orientation; and
determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
12. The non-transitory computer-readable storage medium of claim 11, wherein the prediction comprises data indicative of a probability that the instrument will advance into the first sub-segment.
13. The non-transitory computer-readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause the at least one computing device to:
determining a prediction that the instrument will advance into a second of the sub-segments based on the data indicative of the difference.
14. The non-transitory computer-readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause the at least one computing device to:
determining a third orientation of the instrument at a third time based on third positioning data generated by the set of positioning sensors;
determining an angle between the third orientation and the first orientation;
comparing the angle to a threshold angle; and
in response to the angle being greater than the threshold angle, updating a prediction that the instrument will advance into the first sub-segment.
15. The non-transitory computer-readable storage medium of claim 11, wherein:
the set of positioning sensors includes an Electromagnetic (EM) sensor configured to generate data indicative of an orientation of the EM sensor within an EM field,
the EM field is generated by an EM field generator,
the non-transitory computer-readable storage medium further has stored thereon computer-executable instructions that, when executed, cause the at least one computing device to: determining a yaw angle between the first orientation and the second orientation based on data generated by the EM sensor.
16. The non-transitory computer readable storage medium of claim 15, wherein a yaw axis of the instrument determined based on the EM field is substantially aligned with an orientation difference between a first and second of the subsections.
17. The non-transitory computer-readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause the at least one computing device to:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
18. The non-transitory computer-readable storage medium of claim 11, wherein the positioning data is not registered to a model coordinate system of the model.
19. The non-transitory computer readable storage medium of claim 11, wherein the memory further has stored thereon:
a target path to a target within the model, an
A contralateral registration path comprising: driving the instrument along a first branch of the luminal network outside the target path, returning the instrument to the target path, and driving the instrument along a second branch that is part of the target path,
wherein the non-transitory computer-readable storage medium further has stored thereon instructions that, when executed, cause the at least one computing device to: determining whether the instrument is positioned along the contralateral registration path based on a prediction that the instrument will advance into the first sub-segment.
20. The non-transitory computer-readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause the at least one computing device to:
determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path; and
providing an indication that contralateral registration is unsuccessful in response to determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path.
21. A method of predicting movement of an instrument, comprising:
determining a first orientation of an instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data indicative of a position of the instrument in a positioning sensor coordinate system at a first time;
determining a second orientation of the instrument at a second time based on second positioning data generated by a set of positioning sensors, a distal end of the instrument being located within a first segment of a model at the first and second times and the first segment branching into two or more sub-segments, the model being stored in a memory and modeling a luminal network of a patient;
determining data indicative of a difference between the first orientation and the second orientation; and
determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
22. The method of claim 21, wherein the prediction comprises data indicative of a probability that the instrument will advance into the first sub-segment.
23. The method of claim 21, further comprising:
determining a prediction that the instrument will advance into a second of the sub-segments based on the data indicative of the difference.
24. The method of claim 21, further comprising:
determining a third orientation of the instrument at a third time based on third positioning data generated by the set of positioning sensors;
determining an angle between the third orientation and the first orientation;
comparing the angle to a threshold angle; and
in response to the angle being greater than the threshold angle, updating a prediction that the instrument will advance into the first sub-segment.
25. The method of claim 21, wherein:
the set of positioning sensors includes an Electromagnetic (EM) sensor configured to generate data indicative of an orientation of the EM sensor within an EM field,
the EM field is generated by an EM field generator,
the method further comprises the following steps: determining a yaw angle between the first orientation and the second orientation based on data generated by the EM sensor.
26. The method of claim 25, wherein a yaw axis of the instrument determined based on the EM field is substantially aligned with an orientation difference between a first and second of the subsections.
27. The method of claim 21, further comprising:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
28. The method of claim 21, wherein the positioning data is not registered to a model coordinate system of the model.
29. The method of claim 21, wherein the memory further has stored thereon:
a target path to a target within the model, an
A contralateral registration path comprising: driving the instrument along a first branch of the luminal network outside the target path, returning the instrument to the target path, and driving the instrument along a second branch that is part of the target path,
wherein the method further comprises: determining whether the instrument is positioned along the contralateral registration path based on a prediction that the instrument will advance into the first sub-segment.
30. The method of claim 21, further comprising:
determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path; and
providing an indication that contralateral registration is unsuccessful in response to determining that the instrument is advanced along the target path prior to being advanced along the contralateral registration path.
31. A system, comprising:
a processor; and
at least one computer-readable memory in communication with the processor and having stored thereon a model of a luminal network of a patient, the memory further having stored thereon computer-executable instructions for causing the processor to:
determining an orientation of an instrument relative to the model based on positioning data generated by a set of one or more positioning sensors for the instrument, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments;
determining an orientation of a first of the subsections; and
determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
32. The system of claim 31, wherein the prediction comprises data indicative of a probability that the instrument will advance into the first sub-segment.
33. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment.
34. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining an orientation of a second one of the subsections; and
determining a prediction that the instrument will advance into the second sub-segment based on the orientation of the instrument and the orientation of the second sub-segment.
35. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining an angle between an orientation of the instrument and an orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on the angle.
36. The system of claim 31, wherein the positioning data is registered to a model coordinate system of the model.
37. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
38. The system of claim 37, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a location of a distal end of the instrument relative to the model based on the location data; and
determining an auxiliary prediction that the instrument will advance into the first sub-segment based on a location of a distal end of the instrument,
wherein determining the position of the distal end of the instrument relative to the model is further based on an analysis of the auxiliary prediction.
39. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining that the instrument will advance into the prediction in the first sub-segment when the distal end of the instrument advances from the beginning of the first segment to the end of the first segment.
40. The system of claim 31, wherein the memory has further stored thereon computer-executable instructions for causing the processor to:
determining a prediction that the instrument will advance from the first segment into the first sub-segment independent of a location of a distal end of the instrument within the first segment.
41. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:
determining an orientation of an instrument relative to a model based on positioning data generated by a set of one or more positioning sensors for the instrument, the model being stored in a memory and modeling a luminal network of a patient, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments;
determining an orientation of a first of the subsections; and
determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
42. The non-transitory computer readable storage medium of claim 41, wherein the prediction includes data indicative of a probability that the instrument will advance into the first sub-segment.
43. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment.
44. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining an orientation of a second one of the subsections; and
determining a prediction that the instrument will advance into the second sub-segment based on the orientation of the instrument and the orientation of the second sub-segment.
45. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining an angle between an orientation of the instrument and an orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on the angle.
46. The non-transitory computer readable storage medium of claim 41, wherein the positioning data is registered to a model coordinate system of the model.
47. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
48. The non-transitory computer-readable storage medium of claim 47, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining a location of a distal end of the instrument relative to the model based on the location data; and
determining an auxiliary prediction that the instrument will advance into the first sub-segment based on a location of a distal end of the instrument,
wherein determining the position of the distal end of the instrument relative to the model is further based on an analysis of the auxiliary prediction.
49. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining that the instrument will advance into the prediction in the first sub-segment when the distal end of the instrument advances from the beginning of the first segment to the end of the first segment.
50. The non-transitory computer-readable storage medium of claim 41, further having stored thereon instructions that, when executed, cause at least one computing device to:
determining a prediction that the instrument will advance from the first segment into the first sub-segment independent of a location of a distal end of the instrument within the first segment.
51. A method of predicting movement of an instrument, comprising:
determining an orientation of the instrument relative to a model based on positioning data generated by a set of one or more positioning sensors for the instrument, the model being stored in a memory and modeling a luminal network of a patient, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments;
determining an orientation of a first of the subsections; and
determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
52. The method of claim 51, wherein the prediction comprises data indicative of a probability that the instrument will advance into the first sub-segment.
53. The method of claim 51, further comprising:
determining data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on data indicative of a difference between the orientation of the instrument and the orientation of the first sub-segment.
54. The method of claim 51, further comprising:
determining an orientation of a second one of the subsections; and
determining a prediction that the instrument will advance into the second sub-segment based on the orientation of the instrument and the orientation of the second sub-segment.
55. The method of claim 51, further comprising:
determining an angle between an orientation of the instrument and an orientation of the first sub-segment,
wherein the prediction that the instrument will advance into the first sub-segment is further based on the angle.
56. The method of claim 51, wherein the positioning data is registered to a model coordinate system of the model.
57. The method of claim 51, further comprising:
determining a position of a distal end of the instrument relative to the model by analyzing the positioning data, a set of commands provided to control movement of the instrument, and a prediction that the instrument will advance into the first sub-segment.
58. The method of claim 57, further comprising:
determining a location of a distal end of the instrument relative to the model based on the location data; and
determining an auxiliary prediction that the instrument will advance into the first sub-segment based on a location of a distal end of the instrument,
wherein determining the position of the distal end of the instrument relative to the model is further based on an analysis of the auxiliary prediction.
59. The method of claim 51, further comprising:
determining a prediction that the instrument will advance into the first sub-segment as the distal end of the instrument advances from the beginning of the first segment to the end of the first segment.
60. The method of claim 51, further comprising:
determining a prediction that the instrument will advance from the first segment into the first sub-segment independent of a location of a distal end of the instrument within the first segment.
Technical Field
The systems and methods disclosed herein relate to branch prediction in luminal networks, and more particularly to techniques for predicting into which branch an instrument will advance based on positioning sensor data.
Background
Medical procedures such as endoscopy (e.g., bronchoscopy) may involve the insertion of medical tools into a luminal network (e.g., airway) of a patient for diagnostic and/or therapeutic purposes. Surgical robotic systems may be used to control insertion and/or manipulation of medical tools during medical procedures. The surgical robotic system may include at least one robotic arm including a manipulator assembly that may be used to control placement of a medical tool before and during a medical procedure. The surgical robotic system may also include positioning sensor(s) configured to generate positioning data indicative of a position of a distal end of the medical tool.
The surgical robotic system may also include one or more displays for providing an indication to the user of the location of the distal end of the instrument and thereby assisting the user in navigating the instrument through the patient's luminal network. The system may be configured to perform various techniques that support navigation of the instrument, including predicting which branch of the luminal network the instrument is most likely to proceed from the current branch.
Disclosure of Invention
The systems, methods, and devices of the present disclosure each have several innovative aspects, none of which are solely responsible for the desirable attributes disclosed herein.
In one aspect, a system is provided that includes a processor and at least one computer-readable memory in communication with the processor and having stored thereon a model of a luminal network of a patient, the memory further having stored thereon computer-executable instructions for causing the processor to: determining a first orientation of the instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data being indicative of a position of the instrument in a positioning sensor coordinate system at a first time; determining a second orientation of the instrument at a second time based on second positioning data generated by the set of positioning sensors, the distal end of the instrument being located within a first segment of the model at the first time and the second time and the first segment branching into two or more sub-segments; determining data indicative of a difference between the first orientation and the second orientation; and determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
In another aspect, a non-transitory computer-readable storage medium is provided having stored thereon instructions that, when executed, cause at least one computing device to: determining a first orientation of the instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data being indicative of a position of the instrument in a positioning sensor coordinate system at a first time; determining a second orientation of the instrument at a second time based on second positioning data generated by the set of positioning sensors, the distal end of the instrument being located within a first segment of a model at the first and second times and the first segment branching into two or more sub-segments, the model being stored in the memory and modeling a luminal network of the patient; determining data indicative of a difference between the first orientation and the second orientation; and determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
In yet another aspect, a method of predicting movement of an instrument is provided, comprising: determining a first orientation of the instrument based on first positioning data generated by a set of one or more positioning sensors for the instrument, the first positioning data being indicative of a position of the instrument in a positioning sensor coordinate system at a first time; determining a second orientation of the instrument at a second time based on second positioning data generated by the set of positioning sensors, the distal end of the instrument being located within a first segment of a model at the first and second times and the first segment branching into two or more sub-segments, the model being stored in the memory and modeling a luminal network of the patient; determining data indicative of a difference between the first orientation and the second orientation; and determining a prediction that the instrument will advance into a first of the sub-segments based on the data indicative of the difference.
In yet another aspect, a system is provided that includes a processor and at least one computer-readable memory in communication with the processor and having stored thereon a model of a luminal network of a patient, the memory further having stored thereon computer-executable instructions for causing the processor to: determining an orientation of the instrument relative to the model based on positioning data generated by a set of one or more positioning sensors for the instrument, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments; determining an orientation of a first one of the subsections; and determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
In another aspect, a non-transitory computer-readable storage medium is provided having stored thereon instructions that, when executed, cause at least one computing device to: determining an orientation of the instrument relative to a model stored in a memory and modeling a luminal network of the patient based on positioning data generated by a set of one or more positioning sensors for the instrument, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments; determining an orientation of a first one of the subsections; and determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
In yet another aspect, a method of predicting movement of an instrument is provided, comprising: determining an orientation of the instrument relative to a model stored in a memory and modeling a luminal network of the patient based on positioning data generated by a set of one or more positioning sensors for the instrument, a distal end of the instrument being located within a first segment of the model and the first segment branching into two or more sub-segments; determining an orientation of a first one of the subsections; and determining a prediction that the instrument will advance into the first sub-segment based on the orientation of the instrument and the orientation of the first sub-segment.
Drawings
The disclosed aspects will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like numerals denote like elements.
Fig. 1 shows an embodiment of a cart-based robotic system arranged for use in diagnostic and/or therapeutic bronchoscopy procedures.
Fig. 2 depicts further aspects of the robotic system of fig. 1.
Fig. 3 shows an embodiment of the robotic system of fig. 1 arranged for ureteroscopy.
Fig. 4 shows an embodiment of the robotic system of fig. 1 arranged for vascular surgery.
Fig. 5 shows an embodiment of a table-based robotic system arranged for bronchoscopy procedures.
Fig. 6 provides an alternative view of the robotic system of fig. 5.
FIG. 7 illustrates an example system configured to stow a robotic arm(s).
Fig. 8 illustrates an embodiment of a table-based robotic system configured for a ureteroscopy procedure.
FIG. 9 illustrates an embodiment of a table-based robotic system configured for laparoscopic procedures.
Fig. 10 illustrates an embodiment of the table-based robotic system of fig. 5-9 with pitch or tilt adjustments.
Fig. 11 provides a detailed illustration of the interface between the stage and the column of the stage-based robotic system of fig. 5-10.
Fig. 12 illustrates an example instrument driver.
Fig. 13 illustrates an exemplary medical instrument having a mating instrument driver.
Fig. 14 shows an alternative design of the instrument driver and instrument, in which the axis of the drive unit is parallel to the axis of the elongate shaft of the instrument.
Fig. 15 depicts a block diagram illustrating a positioning system that estimates a position of one or more elements of the robotic system of fig. 1-10 (e.g., a position of an instrument of fig. 13 and 14), according to an example embodiment.
FIG. 16A illustrates an example operating environment implementing one or more aspects of the disclosed branch prediction systems and techniques.
FIG. 16B illustrates an example lumen network that may be navigated in the operating environment of FIG. 16A.
FIG. 16C illustrates an example command console that can be used, for example, as a command console in an example operating environment.
FIG. 17A provides an overview of an example block diagram of a navigation configuration system, according to one embodiment.
FIG. 17B illustrates an example block diagram of the navigation module shown in FIG. 17A in accordance with one embodiment.
FIG. 17C illustrates an example block diagram of an estimated state data store included in a state estimator in accordance with one embodiment.
FIG. 17D illustrates an example location sensor-based branch prediction system in accordance with aspects of the present disclosure.
Fig. 18A is a flow diagram illustrating an example method operable by a robotic system or component(s) thereof for unregistered positioning sensor-based branch prediction, in accordance with aspects of the present disclosure.
Fig. 18B illustrates an example set of localization data points that may be generated by one or more localization sensors as an instrument is driven through a luminal network in accordance with aspects of the present disclosure.
Fig. 19 shows an example lumen network that may perform location sensor-based branch prediction, in accordance with aspects of the present disclosure.
Fig. 20 is a flow diagram illustrating an example method operable by a robotic system or component(s) thereof for registration-based positioning sensor branch prediction, in accordance with aspects of the present disclosure.
Fig. 21 is a flow diagram illustrating an example method operable by a robotic system or component(s) thereof for positioning sensor-based branch prediction, in accordance with aspects of the present disclosure.
Detailed Description
1. Overview.
Aspects of the present disclosure may be integrated into a robotically enabled medical system capable of performing a variety of medical procedures, including both minimally invasive procedures, such as laparoscopy, and non-invasive procedures, such as endoscopy. During endoscopy, the system can perform bronchoscopy, ureteroscopy, gastroscopy, and the like.
In addition to performing a broad range of procedures, the system may provide additional benefits, such as enhanced imaging and guidance to assist the physician. In addition, the system may provide the physician with the ability to perform procedures from an ergonomic position without requiring awkward arm movements and positions. Still further, the system may provide a physician with the ability to perform a procedure with improved ease of use such that one or more instruments of the system may be controlled by a single user.
For purposes of illustration, various embodiments are described below in conjunction with the following figures. It should be understood that many other implementations of the disclosed concept are possible and that various advantages can be achieved with the disclosed implementations. Headings are included herein for reference and to aid in locating the various sections. These headings are not intended to limit the scope of the concepts described with respect to the headings. These concepts may have applicability throughout the present specification.
A. Robotic system-cart.
The robotically enabled medical system may be configured in various ways depending on the particular procedure. Fig. 1 shows an embodiment of a cart-based robot-enabled system 10 arranged for diagnostic and/or therapeutic bronchoscopy procedures. During bronchoscopy, the system 10 may include a
With continued reference to fig. 1, once the
The endoscope 13 may be guided along the patient's trachea and lungs after insertion using precise commands from the robotic system until the target destination or surgical site is reached. To enhance navigation through the patient's pulmonary network and/or to a desired target, the endoscope 13 can be manipulated to telescopically extend the inner guide member portion from the outer sheath portion to achieve enhanced engagement and a larger bend radius. The use of a separate instrument driver 28 also allows the guide portion and sheath portion to be driven independently of each other.
For example, the endoscope 13 may be guided to deliver a biopsy needle to a target, e.g., a lesion or nodule within a patient's lung. A needle may be deployed along a working channel extending along the length of the endoscope to obtain a tissue sample to be analyzed by a pathologist. Depending on the pathology results, additional tools may be deployed along the working channel of the endoscope for additional biopsies. After identifying that the nodule is malignant, the endoscope 13 may endoscopically deliver a tool to resect the potentially cancerous tissue. In some cases, diagnostic and therapeutic treatments may need to be provided in separate processes. In those cases, endoscope 13 may also be used to deliver fiducials to also "mark" the location of the target nodule. In other cases, the diagnostic and therapeutic treatments may be provided during the same procedure.
The system 10 may also include a movable tower 30, which movable tower 30 may be connected to the
To support the above-described robotic system, the tower 30 may include component(s) of a computer-based control system that stores computer program instructions, for example, within a non-transitory computer-readable storage medium such as a permanent magnetic storage drive, a solid state drive, or the like. Execution of these instructions, whether occurring in the tower 30 or in the
The tower 30 may also include pumps, flow meters, valve controllers, and/or fluid inlets to provide controlled irrigation and aspiration capabilities to the system that may be deployed through the endoscope 13. These components may also be controlled using the computer system of the tower 30. In some embodiments, irrigation and aspiration capabilities may be provided directly to endoscope 13 through separate cable(s).
The tower 30 may include a voltage and surge protector designed to provide filtered and protected power to the
The tower 30 may also include support equipment for sensors deployed throughout the robotic system 10. For example, the tower 30 may include optoelectronic devices for detecting, receiving, and processing data received from optical sensors or cameras throughout the robotic system 10. In conjunction with the control system, such optoelectronic devices may be used to generate real-time images for display in any number of consoles deployed throughout the system (including in the tower 30). Similarly, the tower 30 may also include an electronics subsystem for receiving and processing signals received from deployed Electromagnetic (EM) sensors. The tower 30 may also be used to house and position an EM field generator for detection by EM sensors in or on the medical instrument.
The tower 30 may include a console 31 in addition to other consoles available in the rest of the system (e.g., a console mounted on top of the cart). The console 31 may include a user interface and display screen, such as a touch screen, for the physician operator. The console in system 10 is typically designed to provide pre-operative and real-time information for robotic control and procedures, such as navigation and positioning information for endoscope 13. When the console 31 is not the only console available to the physician, the console 31 may be used by a second operator, such as a nurse, to monitor the patient's health or vital signs and operation of the system, as well as to provide process specific data, e.g., navigation and positioning information.
The tower 30 may be coupled to the
Fig. 2 provides a detailed illustration of an embodiment of a cart of the cart-based robot-enabled system shown in fig. 1. The
The carriage interface 19 is connected to the column 14 by slots, such as
In some embodiments, the
The column 14 may internally include mechanisms such as gears and motors designed to use vertically aligned lead screws to mechanically translate the carriage 17 in response to control signals generated in response to user inputs (e.g., inputs from the console 16).
The robotic arm 12 may generally include a
The cart base 15 balances the weight of the column 14, carriage 17 and arm 12 on the floor. Thus, the cart base 15 houses heavier components such as electronics, motors, power supplies, and components that enable the cart to be moved and/or secured. For example, the cart base 15 includes rollable wheel casters 25 that allow the cart to be easily moved in the chamber prior to a procedure. After reaching the proper position, the caster 25 may be secured using a wheel lock to hold the
The console 16, disposed at the vertical end of the column 14, allows both a user interface for receiving user inputs and a display screen (or dual-purpose device, such as a touch screen 26) to provide both pre-operative and intra-operative data to the physician user. The potential preoperative data on the touchscreen 26 may include navigation and mapping data derived from a preoperative Computerized Tomography (CT) scan, preoperative planning, and/or annotations from preoperative patient interviews. The intra-operative data on the display may include optical information provided from the tool, sensor and coordinate information from the sensors, and vital patient statistics such as respiration, heart rate, and/or pulse. The console 16 may be positioned and tilted to allow the physician access to the console from the side of the column 14 opposite the cradle 17. From this position, the physician can view the console 16, the robotic arm 12, and the patient while manipulating the console 16 from behind the
Fig. 3 shows an embodiment of a robot-enabled system 10 arranged for ureteroscopy. During ureteroscopy, the
After insertion into the urethra, ureteroscope 32 may be navigated to the bladder, ureter, and/or kidney for diagnostic and/or therapeutic applications using similar control techniques as in bronchoscopy. For example, the ureteroscope 32 may be guided into the ureter and kidney to break up accumulated kidney stones using a laser or ultrasonic lithotripsy device deployed along the working channel of the ureteroscope 32. After lithotripsy is complete, the resulting stone fragments may be removed using a basket deployed along the ureteroscope 32.
Fig. 4 shows an embodiment of a similarly arranged robot-enabled system for vascular surgery. In vascular procedures, the system 10 may be configured such that the
B. Robotic system-station.
Embodiments of the robot-enabled medical system may also incorporate a patient table. The incorporation of a table reduces the amount of capital equipment in the operating room by removing the cart, which allows for better access to the patient. Fig. 5 shows an embodiment of such a robot-enabled system arranged for a bronchoscopy procedure. The
Fig. 6 provides an alternative view of the
The
The
The
With continued reference to fig. 6, the
In some embodiments, the table base may stow and store the robotic arm when not in use. FIG. 7 illustrates a system 47 for stowing a robotic arm in an embodiment of the table-based system. In the system 47, the carriage 48 may be vertically translated into the base 49 to stow the robotic arm 50, arm mount 51, and carriage 48 within the base 49. The base cover 52 may be translated and retracted to open to deploy the carriage 48, arm mount 51, and arm 50 around the post 53, and to close to stow to protect the carriage, arm mount, and arm when not in use. The base cover 52 may be sealed with a membrane 54 along the edges of the base cover opening to prevent the ingress of dust and fluids when closed.
Fig. 8 illustrates an embodiment of a robot-enabled table-based system configured for a ureteroscopy procedure. In ureteroscopy, table 38 may include a rotating portion 55 for positioning the patient at an angle off of
During laparoscopic procedures, minimally invasive instruments (elongated in shape to accommodate the size of one or more incisions) may be inserted into the patient's anatomy through a small incision(s) in the patient's abdominal wall. After the patient's abdominal cavity is inflated, instruments commonly referred to as laparoscopes may be guided to perform surgical tasks such as grasping, cutting, ablating, suturing, and the like. FIG. 9 illustrates an embodiment of a robot-enabled table-based system configured for laparoscopic procedures. As shown in fig. 9, the carriage 43 of the
To accommodate laparoscopic procedures, the robotic enabling table system may also tilt the platform to a desired angle. Fig. 10 illustrates an embodiment of a robot-enabled medical system with pitch or tilt adjustment. As shown in fig. 10,
Fig. 11 provides a detailed illustration of the interface between table 38 and
For example, pitch adjustment is particularly useful when attempting to position the table in a head-low-foot (Trendelenburg) position (i.e., the patient's lower abdomen is at a higher position from the ground than the patient's lower abdomen) for lower abdominal surgery. The low head and foot position allows the patient's internal organs to slide by gravity toward his/her upper abdomen, thereby emptying the abdominal cavity for access by minimally invasive tools and performing lower abdominal surgical procedures, such as laparoscopic prostatectomy.
C. An instrument driver and interface.
An end effector of a robotic arm of the system comprises: (i) an instrument driver (alternatively referred to as an "instrument drive mechanism" or "instrument device manipulator") that incorporates an electromechanical device for actuating a medical instrument; and (ii) a removable or detachable medical instrument that may be free of any electromechanical components such as a motor. This dichotomy may be driven by the need to sterilize medical instruments used in medical procedures and the inability to adequately sterilize expensive capital equipment due to the complex mechanical components and sensitive electronics of the expensive capital equipment. Thus, the medical instrument may be designed to be detached, removed, and interchanged from the instrument driver (and thus from the system) for individual sterilization or disposal by the physician or the physician's staff. In contrast, the instrument driver does not need to be changed or sterilized and may be covered with a drape for protection.
FIG. 12 illustrates an example instrument driver. The
For procedures requiring a sterile environment, the robotic system may incorporate a drive interface between the instrument driver and the medical instrument, such as a sterile adapter connected to a sterile drape (drapee). The primary purpose of the sterile adapter is to transfer angular motion from the drive shaft of the instrument driver to the drive input of the instrument while maintaining a physical separation between the drive shaft and the drive input and thus maintaining sterility. Thus, an example sterile adapter may include a series of rotational inputs and outputs intended to mate with a drive shaft of a device driver and a drive input on a device. A sterile drape (e.g., transparent or translucent plastic) composed of a thin, flexible material connected to a sterile adaptor is designed to cover capital equipment such as instrument drivers, robotic arms and carts (in cart-based systems) or tables (in table-based systems). Use of the drape will allow capital equipment to be placed near the patient while still being located in areas that do not require sterilization (i.e., non-sterile areas). On the other side of the sterile drape, the medical device may be docked with the patient in the area requiring sterilization (i.e., the sterile field).
D. A medical device.
FIG. 13 illustrates an example medical instrument having a mating instrument driver. Similar to other instruments designed for use with robotic systems, the medical instrument 70 includes an elongate shaft 71 (or elongate body) and an instrument base 72. The instrument base 72, also referred to as an "instrument handle" due to its intended design for manual interaction by a physician, may generally include a rotatable drive input 73, such as a socket, pulley, or reel, designed to mate with a drive output 74 extending through a drive interface on an instrument driver 75 at the distal end of a
Elongate shaft 71 is designed to be delivered through an anatomical opening or lumen (e.g., in endoscopy) or through a minimally invasive incision (e.g., in laparoscopy). The
Torque from instrument driver 75 is transmitted along elongate shaft 71 using tendons within shaft 71. These separate tendons (e.g., pull wires) may be individually anchored to separate drive inputs 73 within the instrument handle 72. Tendons are guided from handle 72 along one or more traction lumens within elongate shaft 71 and anchored at a distal portion of elongate shaft 71. In laparoscopy, these tendons may be coupled to a distally mounted end effector, such as a wrist, grasper, or scissors. With such an arrangement, torque applied to drive input 73 transfers tension to the tendons, causing the end effector to actuate in some manner. In laparoscopy, the tendon may rotate a joint about an axis, thereby moving an end effector in one direction or the other. Alternatively, the tendon may be connected to one or more jaws (jaw) of the grasper at the distal end of the elongate shaft 71, wherein tension from the tendon closes the grasper.
In endoscopy, the tendons can be coupled to bending or engaging sections disposed along (e.g., at the distal end of) elongate shaft 71 via adhesives, control rings, or other mechanical fasteners. When fixedly attached to the distal end of the bending section, the torque applied to the drive input 73 will be transmitted along the tendon, bending or engaging the softer bending section (sometimes referred to as the engageable section or region). Along the non-curved section, it may be advantageous to spiral or spiral the separate traction lumens guiding the separate tendons along (or inside) the wall of the endoscope shaft to balance the radial forces caused by the tension in the traction wires. The spacing therebetween and/or the angle of the spirals may be varied or designed for a particular purpose, wherein a tighter spiral exhibits less shaft compression under load forces, and a lesser amount of the spiral causes greater shaft compression under load forces, but also exhibits limited bending. In another aspect, the distraction lumen can be oriented parallel to the longitudinal axis of elongate shaft 71 to allow controlled engagement in a desired curved or engageable section.
In endoscopy, the elongate shaft 71 houses a number of components to assist in robotic procedures. The shaft may include a working channel for deploying surgical tools, irrigation and/or suction to a surgical field at the distal end of the shaft 71. The shaft 71 may also house wires and/or optical fibers to transmit signals to/from an optical assembly at the distal tip, which may include an optical camera. The shaft 71 may also house optical fibers to transmit light from a light source (e.g., a light emitting diode) located at the proximal end to the distal end of the shaft.
At the distal end of the instrument 70, the distal tip may also include openings for working channels for delivering tools for diagnosis and/or treatment, irrigation, and aspiration to the surgical site. The distal tip may also include a port for a camera device, such as a fiberscope or digital camera, to capture images of the internal anatomical space. Relatedly, the distal tip may also include a port for a light source to illuminate the anatomical space when the camera device is in use.
In the example of fig. 13, the drive shaft axis, and thus the drive input axis, is orthogonal to the axis of the elongate shaft. However, this arrangement complicates the rolling ability of elongate shaft 71. As the tendons extend away from drive input 73 and into a traction lumen within elongate shaft 71, rolling elongate shaft 71 along its axis while keeping drive input 73 stationary can cause undesirable tangling of the tendons. Such eventual entanglement of the tendons may disrupt any control algorithms intended to predict movement of the flexible elongate shaft during the endoscopic procedure.
Fig. 14 shows an alternative design of the instrument driver and instrument, in which the axis of the drive unit is parallel to the axis of the elongate shaft of the instrument. As shown, the circular instrument driver 80 includes four drive units whose drive outputs 81 are aligned in parallel at the end of a robotic arm 82. The drive units and their respective drive outputs 81 are housed in a rotation assembly 83 of the instrument driver 80 driven by one of the drive units within the assembly 83. In response to the torque provided by the rotational drive unit, the rotation assembly 83 rotates along a circular bearing that connects the rotation assembly 83 to the non-rotating portion 84 of the instrument driver. Power and control signals may be transmitted from the non-rotating portion 84 of the instrument driver 80 to the rotating assembly 83 through electrical contacts that may be maintained by rotation of a brush-slip ring connection (not shown). In other embodiments, the rotational assembly 83 may be responsive to a separate drive unit integrated into the non-rotating portion 84, and thus not parallel to the other drive units. Rotation mechanism 83 allows instrument driver 80 to rotate the drive unit and its corresponding drive output 81 as a single unit about instrument driver axis 85.
Similar to the earlier disclosed embodiments, the instrument 86 may include an elongated shaft portion 88 and an instrument base 87 (shown in transparent appearance for discussion purposes), the instrument base 87 including a plurality of drive inputs 89 (e.g., sockets, pulleys, and spools) configured to receive the drive outputs 81 in the instrument driver 80. Unlike the previously disclosed embodiment, the instrument shaft 88 extends from the center of the instrument base 87 with its central axis substantially parallel to the axis of the drive input 89, rather than orthogonal as in the design of fig. 13.
When coupled to the rotation assembly 83 of the instrument driver 80, a medical instrument 86 including an instrument base 87 and an instrument shaft 88 rotates in conjunction with the rotation assembly 83 about an instrument driver axis 85. Since the instrument shaft 88 is centered in the instrument base 87, the instrument shaft 88 is coaxial with the instrument driver axis 85 when attached. Thus, rotation of the rotation assembly 83 rotates the instrument shaft 88 about its own longitudinal axis. Furthermore, when instrument base 87 rotates with instrument shaft 88, any tendons connected to drive input 89 in instrument base 87 do not tangle during rotation. Thus, the parallelism of the axes of drive output 81, drive input 89, and instrument shaft 88 allows shaft rotation without tangling any control tendons.
E. Navigation and control.
Conventional endoscopy may include the use of fluoroscopy (e.g., as may be delivered through a C-arm) and other forms of radiation-based imaging modalities to provide intraluminal guidance to the operating physician. In contrast, robotic systems contemplated by the present disclosure may provide non-radiation based navigation and positioning means to reduce physician exposure to radiation and reduce the number of devices in the operating room. As used herein, the term "locating" may refer to determining and/or monitoring the position of an object in a reference coordinate system. Techniques such as pre-operative mapping, computer vision, real-time EM tracking, and robot command data may be used alone or in combination to achieve a radiation-free operating environment. In still other cases where a radiation-based imaging modality is used, preoperative mapping, computer vision, real-time EM tracking, and robot command data may be used, alone or in combination, to improve the information obtained only by the radiation-based imaging modality.
Fig. 15 is a block diagram illustrating a positioning system 90 that estimates a position of one or more elements of a robotic system (e.g., a position of an instrument), according to an example embodiment. The positioning system 90 may be a collection of one or more computer devices configured to execute one or more instructions. The computer apparatus may be implemented by one or more processors and computer readable memory in one or more of the components discussed above. By way of example and not limitation, the computer device may be located in the tower 30 shown in fig. 1, the cart shown in fig. 1-4, the bed shown in fig. 5-10, or the like.
As shown in fig. 15, the localization system 90 may include a localization module 95 that processes the input data 91-94 to generate localization data 96 for the distal tip of the medical instrument. The positioning data 96 may be data or logic representing the position and/or orientation of the distal end of the instrument relative to a frame of reference. The reference frame may be a reference frame relative to the anatomy of the patient or relative to a known object, such as an EM field generator (see discussion below regarding EM field generators). The positioning data 96, which may also be referred to herein as "state data," describes a current state of the distal tip of the medical instrument relative to a model (e.g., a model of a bone) of the patient's anatomy. The status data may include information such as the position and orientation of the distal tip of the medical instrument during a given sampling period. For example, when using a skeletal model to model a patient's anatomy based on the midpoints of the lumen network, the location may take the form of a segment ID and a depth along the segment.
The various input data 91 to 94 will now be described in more detail. The pre-operative mapping may be accomplished by using a set of low dose CT scans. The pre-operative CT scan is reconstructed into a three-dimensional (3D) image that is visualized, for example, as a "slice" of a cross-sectional view of the patient's internal anatomy. When analyzed in its entirety, an image-based model of the anatomical cavities, spaces, and structures for the patient's anatomy (e.g., the patient's lung network) may be generated. Techniques such as centerline geometry may be determined and approximated from the CT images to form a three-dimensional volume of the patient's anatomy referred to as pre-operative model data 91. The use of centerline geometry is discussed in U.S. patent application No. 14/523,760, which is incorporated herein in its entirety. The network topology model can also be derived from CT images and is particularly suitable for bronchoscopy.
In some embodiments, the instrument may be equipped with a camera to provide visual data 92. The positioning module 95 may process the visual data to enable one or more vision-based positioning tracks. For example, the pre-operative model data may be used in conjunction with the vision data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or an instrument advanced through a working channel of an endoscope). For example, using the pre-operative model data 91, the robotic system may generate a library of expected endoscope images from the model based on the expected path of travel of the endoscope, each image linked to a location within the model. During surgery, the robotic system may reference the library to compare real-time images captured at a camera (e.g., at the distal end of the endoscope) with images in the image library to assist in positioning.
Other computer vision based tracking techniques use feature tracking to determine the motion of the camera, and thus the endoscope. Some features of the localization module 95 may identify circular geometries in the pre-operative model data 91 that correspond to anatomical lumens and track changes in those geometries to determine which anatomical lumen was selected, as well as relative rotational and/or translational motion of the camera. The use of a topological map may further enhance the vision-based algorithms or techniques.
Optical flow, another computer vision based technique, may analyze the displacement and translation of image pixels in a video sequence in the visual data 92 to infer camera motion. Examples of optical flow techniques may include motion detection, object segmentation computation, luminance, motion compensated coding, stereo disparity measurement, and so forth. By comparing multiple frames with multiple iterations, the movement and positioning of the camera (and hence the endoscope) can be determined.
The localization module 95 may use real-time EM tracking to generate a real-time localization of the endoscope in a global coordinate system, wherein the global coordinate system may be registered to the patient's anatomy represented by the pre-operative model. In EM tracking, an EM sensor (or tracker), which includes one or more sensor coils embedded in a medical instrument (e.g., an endoscopic tool) at one or more positions and orientations, measures changes in the EM field produced by one or more static EM field generators positioned at known locations. The positioning information detected by the EM sensor is stored as
The positioning module 95 may also use the robot commands and
As shown in fig. 15, the location module 95 may use several other input data. For example, although not shown in fig. 15, an instrument utilizing a shape sensing fiber may provide shape data that the localization module 95 may use to determine the position and shape of the instrument.
The positioning module 95 may use the input data 91 to 94 in combination. In some cases, such a combination may use a probabilistic approach, where the localization module 95 assigns confidence weights to the locations determined from each of the input data 91 to 94. Thus, in situations where the EM data may be unreliable (as is the case in the presence of EM interference), the confidence in the localization determined by
As discussed above, the robotic systems discussed herein may be designed to incorporate a combination of one or more of the above techniques. A computer-based control system of a robotic system located in a tower, bed, and/or cart may store computer program instructions within a non-transitory computer-readable storage medium, such as a permanent magnetic storage drive, a solid state drive, or the like, wherein the computer program instructions, when executed, cause the system to receive and analyze sensor data and user commands, generate control signals for the entire system, and display navigation and positioning data, such as the position of an instrument within a global coordinate system, anatomical map, or the like.
2. Introduction of branch prediction based on positioning sensors.
Embodiments of the present disclosure relate to systems and techniques for location sensor based branch prediction. The system may employ a localization sensor(s) or localization sensing device(s) to localize, for example, the distal end of the instrument during a medical procedure. The positioning sensor(s) may be positioned at or near the distal end of the instrument, or may be positioned remote from the distal end of the instrument. Examples of a position sensor or position sensing device that may be positioned at or near the distal end of the instrument include an EM sensor, a vision-based position sensor (e.g., a camera device), a shape sensing fiber optic, and the like. Examples of a position sensor or position sensing device that may be disposed remote from the distal end of the instrument include a fluoroscopic imaging device, robotic system component(s) that generate or process robotic data for controlling the position of the instrument via one or more instrument manipulators, a remote vision-based position sensor, and the like.
The positioning sensor may be configured to generate positioning data indicative of a position of the distal end of the instrument (e.g., relative to a positioning sensor coordinate system). As used herein, a positioning sensor coordinate system may refer to any coordinate system that may be used to define or determine the location (e.g., on a manifold such as euclidean space) of positioning data generated by a positioning sensor. When the positioning sensor is juxtaposed to the distal end of the instrument, the positioning data may represent the position of the positioning sensor itself, which the processor may then use to determine the position of the distal end of the instrument. In some embodiments, the positioning sensor coordinate system may include a set of origins and axes, which may be defined based on the particular technology used to implement the positioning sensor.
For example, an EM sensor located in or on the instrument may be configured to measure the EM field generated by the EM field generator. The characteristics of the EM field, and thus the EM values measured by the EM sensors, may be defined with respect to the location and orientation of the EM field generator. Thus, the placement of the EM field generator may affect the values measured by the EM sensors, and may also define the location and orientation of the EM coordinate system.
As described above, the luminal network of the patient can be mapped preoperatively using, for example, a low dose CT scan to generate a model of the luminal network. Because the model may be generated via a different technique than that used to position the distal end of the instrument, the model coordinate system may not be aligned with the positioning sensor coordinate system. Accordingly, to track the position of the instrument relative to the model using the positioning sensor coordinate system, one technique may involve registering the coordinate system used by one or more positioning sensors with another coordinate system (e.g., the coordinate system used by the anatomical model) (e.g., by a separate system communicatively coupled to the robotic system or one or more components of the robotic system, including but not limited to a processor, a positioning system, a positioning module, etc.). The registration may include, for example, a translation and/or rotation applied to the positioning sensor data to map the positioning sensor data from the positioning sensor coordinate system to the model coordinate system.
The system or processor may perform registration of the localization sensor coordinate system to the model coordinate system, for example, during an initial stage of the procedure. According to implementations, the processor may automatically perform the registration process in the background as the instrument is initially advanced through the luminal network. In another implementation, the processor may provide a set of instructions to the user to drive the instrument to a particular location within the luminal network or along a set registration path to facilitate the registration process. Thus, the processor may perform a portion of the process when the position data received from the positioning sensor(s) is not registered to the model.
To provide feedback to the user regarding instrument navigation during a medical procedure, a "fused" localization algorithm may be run (e.g., by the localization system 90 of fig. 15). The fusion algorithm may combine data received from multiple sources indicative of the location of the distal end of the instrument to determine the location of the instrument. One function of the fusion algorithm may also include prediction of the next branch of the luminal network into which the instrument may advance. The prediction may be based on at least some of the data sources used in the fusion algorithm, including the positioning sensor data. In some embodiments, the prediction may also be used as an input in determining the position of the distal end of the instrument.
Since the localization sensor(s) may not be registered for at least a portion of the medical procedure, certain aspects of the present disclosure may relate to techniques for branch prediction, which may be employed based on either the registered localization sensor data or the unregistered localization sensor data (also commonly referred to as "raw" localization sensor data). Thus, the system may selectively apply different techniques or combinations thereof to location sensor-based branch prediction depending on whether the location sensor(s) have been registered.
Em navigation guides bronchoscopy.
In the following, an example system will be described that may employ techniques for location sensor-based branch prediction. For example, the system may be configured for EM navigation guided bronchoscopy procedures. However, aspects of the present disclosure may also be applied to systems using other positioning sensors capable of generating positioning data, as well as other types of medical procedures.
FIG. 16A illustrates an
The
The
Fig. 16B illustrates an example
In some embodiments, a two-dimensional (2D) display of a 3D luminal network model as described herein or a cross-section of the 3D model may be similar to fig. 16B. The estimated position of the distal end of the instrument may be overlaid onto such a representation. In some implementations, the estimated position can be displayed on a display of a command console, such as the
FIG. 16C illustrates an
The
In some embodiments, a model of the
B. Location sensor based branch prediction using unregistered location data.
As described above, an initial phase of the medical procedure may be performed prior to registering the localization sensor(s) to the model of the luminal network. However, the localization sensor(s) may still generate localization data prior to registration. Although not registered to the model, the raw positioning sensor data may be used to provide certain positioning and navigation functions. For example, the processor may determine the relative orientation of the instrument at different times based on the raw, unregistered positioning data. Thus, in certain embodiments, the processor may facilitate predicting the next branch in the luminal network to which the instrument is likely to advance based on the shape and structure of the luminal network and the orientation of the instrument determined based on the unregistered data.
Branch prediction may be included as part of a navigation configuration system. 17A-17D illustrate example block diagrams of a
Input data, as used herein, refers to raw data collected from and/or processed by an input device, e.g., command module(s), optical sensor(s), EM sensor(s), IDM(s), for generating estimated state information of an endoscope and outputting navigation data. The plurality of input data storage devices 210 to 240 include an image data storage device 210, an EM
The output
To determine the output navigation data, the
FIG. 17B illustrates an example block diagram of the
A
Fig. 17C illustrates an example block diagram of an estimated
The respective storage devices described above represent the estimated state data in various ways. Specifically, bifurcation data refers to the positioning of a medical instrument relative to a set of branches (e.g., a bifurcation, a trifurcation, or a bifurcation (division) of more than three branches) within a tubular network. For example, the bifurcation data may be a set of branch choices selected by the instrument as it travels through the tubular network based on a larger set of available branches provided, for example, by a 3D model that maps the entire tubular network. The bifurcation data may also include information ahead of the location of the instrument tip, such as branches (bifurcations) that the instrument tip is approaching but has not yet traveled through, but that have been detected, e.g., based on current position information of the tip relative to a 3D model, or based on captured images of upcoming bifurcations.
The position data indicates a three-dimensional position of a certain part of the medical instrument within the tubular network or of a certain part of the tubular network itself. The position data may be in the form of absolute positioning or relative positioning with respect to the 3D model of the tubular network, for example. As one example, the position data may include an indication that the localized position of the instrument is within a particular branch. The identification of a particular branch may also be stored as a segment Identification (ID) that uniquely identifies the particular segment in the model in which the instrument tip is located.
The depth data is indicative of depth information of the instrument tip within the tubular network. Example depth data includes the total insertion (absolute) depth of the medical instrument within the patient and the (relative) depth within the identified branch (e.g., the segment identified by the location data store 287). The depth data may be determined based on position data about both the tubular network and the medical instrument.
The orientation data is indicative of orientation information of the instrument tip and may include overall roll, pitch, and yaw associated with the 3D model and pitch, roll, and yaw within the identified branches.
Returning to fig. 17B, the
Since the
As used herein, a "probability" in a "probability distribution" refers to the likelihood that an estimate of a possible location and/or orientation of a medical instrument is correct. For example, different probabilities may be calculated by one of the algorithm modules indicating the relative likelihood of the medical instrument being in one of several different possible branches within the tubular network. In one embodiment, the type of probability distribution (e.g., discrete distribution or continuous distribution) is selected to match the characteristics of the estimated states (e.g., the type of estimated state, such as continuous location information and discrete branch selection). As one example, the estimated state used to identify which segment of the trifurcation the medical instrument is in may be represented by a discrete probability distribution and may include three
In contrast, a "confidence value," as used herein, reflects a measure of confidence in an estimate of a state provided by one of the algorithms based on one or more factors. For EM-based algorithms, factors such as EM field distortion, EM registration inaccuracies, patient displacement or movement, and patient respiration may affect the confidence of the estimate of the state. In particular, the confidence value of the estimate of the state provided by the EM-based algorithm may depend on the particular respiratory cycle of the patient, the movement of the patient or the EM field generator, and the location within the anatomy in which the instrument tip is located. For image-based algorithms, example factors that may affect the confidence value when estimating the state include: a lighting condition of a location within the anatomy at which the image was captured; the presence of fluid, tissue or other obstructions against or in front of the optical sensor that captured the image; the patient's breathing; conditions of the patient's own tubular network (e.g., lungs), such as general fluid within the tubular network and occlusion of the tubular network; and specific operating techniques used in, for example, navigation or image capture.
For example, one factor may be that a particular algorithm has different levels of accuracy at different depths in the lungs of a patient, such that in the case of relatively close airway openings, the particular algorithm may have high confidence in its estimation of the position and orientation of the medical instrument, but the medical instrument travels further into the bottom of the lungs, the confidence value may drop. Generally, the confidence value is based on one or more system factors related to the process of determining the outcome, while the probability is a relative measure that occurs when attempting to determine the correct outcome from multiple probabilities using a single algorithm based on the underlying data.
As one example, the mathematical formula for calculating the result of the estimated states represented by a discrete probability distribution (e.g., branch/segment identification through three values of the estimated state involved for trifurcations) may be as follows:
S1=CEM*P1,EM+CImage*P1,Image+CRobot*P1,Robot;
S2=CEM*P2,EM+CImage*P2,Imtage+CRobot*P2,Robot;
S3=CEM*P3,EM+CImage*P3,Image+CRobot*P3,Robot。
in the above exemplary mathematical formula, Si(i ═ 1, 2, 3) represents a possible example value of the estimated state in the case where 3 possible segments are identified or present in the 3D model, CEM、CImageAnd CRobotRepresentation and EM-based algorithms, graphs-basedConfidence values corresponding to the image algorithm and the robot-based algorithm, and Pi,EM、Pi,ImageAnd Pi,RobotRepresenting the probability of segment i.
To better illustrate the concepts of probability distributions and confidence values associated with estimated states, detailed examples are provided herein. In this example, the user is attempting to identify the segment in which the instrument tip is located in a particular trifurcation within the central airway (predicted region) of the tubular network, and uses three algorithm modules, including EM-based algorithms, image-based algorithms, and robot-based algorithms. In this example, the probability distribution corresponding to the EM-based algorithm may be 20% in the first branch, 30% in the second branch, and 50% in the third (last) branch, with a confidence value of 80% applied to the EM-based algorithm and the central airway. For the same example, for the first, second, and third branches, the probability distribution corresponding to the image-based algorithm may be 40%, 20%, 40%, and the confidence value applied to the image-based algorithm is 30%; while for the first, second and third branches, the probability distribution corresponding to the robot-based algorithm may be 10%, 60%, 30% and the confidence value applied to the image-based algorithm is 20%. The difference in confidence values applied to the EM-based algorithm and the image-based algorithm indicates: EM-based algorithms may be a better choice for segment identification in the central airway than image-based algorithms. An example mathematical calculation of the final estimated state may be:
for the first branch: 20% + 80% + 40% + 30% + 10% + 20% + 30%; for the second branch: 30% + 80% + 20% + 30% + 60% + 20% + 42%; and for the third branch: 50% + 80% + 40% + 30% + 20% + 58%.
In this example, the output estimated state of the instrument tip may be the resulting values (e.g., 30%, 42%, and 58% derived) or derived values from these resulting values, such as determining that the instrument tip is in the third branch. Although this example describes the use of algorithm modules including EM-based algorithms, image-based algorithms, and robot-based algorithms, the estimation of the state of the instrument tip may also be provided based on different combinations of various algorithm modules including path-based algorithms.
As described above, the estimated states may be represented in a number of different ways. For example, the estimated state may also include an absolute depth of localization from the airway to the instrument tip and a data set representing a set of branches traversed by the instrument within the tubular network, e.g., a set of branches is a subset of the entire set of branches provided by the 3D model of the patient's lungs. The application of the probability distribution and the confidence values on the estimated state allows to improve the accuracy of the estimation of the position and/or orientation of the instrument tip within the tubular network.
As shown in fig. 17B, the algorithm modules include an EM-based
B.1. Branch prediction system
The EM-based
FIG. 17D illustrates an example location sensor based location and branch prediction system in accordance with aspects of the present disclosure. In particular, fig. 17D illustrates a localized sensor-based localization and branch prediction system that includes
The EM
B.2. Example route taken by an Instrument
For illustrative purposes, various aspects of the present disclosure relating to location sensor-based branch prediction will be described in the context of bronchoscopy and the navigation portion of a bronchial lumen network. However, the present disclosure may also be applied to other luminal networks and other medical procedures.
Fig. 18A illustrates an example lumen network in which location sensor-based branch prediction may be performed in accordance with various aspects of the present disclosure. In the embodiment of fig. 18A, the illustrated network of lumens 300 corresponds to airways of a patient's lungs and includes a first-generation airway 305 (e.g., trachea) that branches into two second-generation airways 315 and 320 (e.g., a main bronchus) at a main carina 310. Assuming the patient lies supine, trachea 305 will branch into the patient's left bronchus 320 and the patient's right bronchus 315.
Fig. 18A also shows a route 325 along which the instrument may be driven as it is navigated through the airway during a medical procedure. Two example tracking positions 330 and 335 of the instrument as driven along the path 325 are shown and will be referenced in discussing various embodiments. The position sensor may generate position data (not shown) representing the tracking position 330 and the tracking position 335 within the position sensor coordinate system, as described in detail below in connection with fig. 18B. Various aspects of the present disclosure may relate to the prediction of which airway between the second generation airway 315 and one of the second generation airways 320 the instrument will advance to. As will be described in detail later, the processor may select an initial position 330 and a subsequent position 335 of the instrument for use in a branch prediction technique.
B.3. Example positioning data generated during a procedure
Fig. 18B illustrates an example set of localization data points that may be generated by one or more localization sensors as an instrument is driven through a luminal network in accordance with various aspects of the present disclosure. In fig. 18B, the
During the procedure, the localization sensor may generate a plurality of
B.4. Example Branch prediction techniques
Fig. 19 is a flow diagram illustrating an example method operable by a robotic system or component(s) thereof for location sensor-based branch prediction in accordance with various aspects of the present disclosure. For example, the steps of
At
At
At
In one implementation, the positioning sensor-based branch prediction system may determine the difference between the initial orientation and the subsequent orientation by calculating a relative transformation matrix between the initial orientation and the subsequent orientation. The system may decompose the transformation matrix into roll, pitch, and yaw angles to define the orientation of the instrument in the position sensor coordinate system. Certain localization sensor technologies (e.g., EM localization sensors) that may be used as localization sensors may be substantially aligned with a patient in at least one angular degree of freedom when the patient is supine to perform a given procedure (e.g., to perform a bronchoscopy procedure). In an EM implementation, the EM field generator may generate an EM field having an orientation defined relative to an orientation of the EM field generator. Thus, by arranging the orientation of the EM field generator to be aligned with the patient, the system is able to perform the
Since the EM field and the orientation of the patient may be known for the bronchoscopy procedure, the system is able to determine a yaw angle between the initial orientation and the subsequent orientation based on data generated by the EM sensor. Thus, the bifurcation of the trachea to the main bronchus may be defined relative to the yaw axis in the EM sensor coordinate system. Thus, when the instrument is positioned within trachea 305 (see fig. 18A), the yaw angle of the instrument determined based on the EM field may be substantially aligned with the difference in orientation between main bronchus 315 and main bronchus 320. Thus, the system may use the change in yaw angle between the initial orientation and the subsequent orientation as a basis for updating the prediction of the main bronchus 315 and the main bronchus 320 into which the instrument may be advanced.
In some embodiments, block 415 may further include: the system determines the angle formed between the orientations of the sub-segments. The amount of change in the orientation of the distal end of the instrument may correspond to the angle formed between the sub-segments in order to redirect the insertion direction of the instrument from one sub-segment to the other. As described below, the system may use the angle formed between the subsections as a factor in determining whether to update branch predictions and/or as a factor in assigning probabilities to subsections during branch prediction. It should be understood that embodiments in which the threshold is selected based on the angle formed between the sub-segments may be determined at design time (e.g., the threshold is a parameter selected by a designer of the system based on the angle) or during runtime (e.g., the system includes logic and data for dynamically determining the threshold before or during surgery). It should also be appreciated that the runtime method may affect the hardware and software of the system as well as the patient's anatomy to accurately distinguish the angles using the threshold.
At
In embodiments where the system determines angles between sub-segments, the system may use the angles formed between the orientations of the sub-segments in determining the predictions. In other embodiments, the system may determine the prediction based on a difference between an orientation of the distal end of the instrument and an orientation of each of the sub-segments. In one embodiment, the system may assign a higher probability to a sub-segment having a smaller orientation difference from the orientation of the instrument than the remaining sub-segments.
In certain embodiments, the system may avoid adjusting the probability of the instrument advancing into each of the sub-segments unless the orientation of the instrument has changed beyond a threshold level. In some embodiments, the system may adjust the threshold level based on the angle between the sub-segments. For example, the system may increase the threshold level when the angle between the sub-segments is greater than a first threshold angle, and may decrease the threshold level when the angle between the sub-segments is less than a second threshold angle. Once the system has determined a prediction that the instrument will advance to a given one of the sub-segments (e.g., based on a probability of advancing to the given sub-segment being greater than a probability of other sub-segments), the system may then determine an orientation of the instrument at a third time based on subsequent positioning data generated by the set of positioning sensors. The system may be configured to: calculating an angle between (i) an orientation at a third time (e.g., a time after the second time when the instrument is at a subsequent position) and (ii) the initial orientation; and comparing the calculated angle to a threshold angle value. The system may also be configured to update the probability of the instrument advancing into each of the sub-segments in response to the calculated angle being greater than the threshold angle value. Thus, in certain embodiments, the system does not update the probability of the instrument advancing into each of the sub-segments unless the orientation of the instrument makes an angle with the initial orientation that is greater than a threshold angle value. The
The above-described
In one example, when the data indicative of the difference between the initial orientation and the subsequent orientation coincides with the instrument being advanced toward or pointed at the patient's left main bronchus (e.g., second-generation airway 320), the system may predict that the instrument will be advanced into the left main bronchus instead of the right main bronchus. The system may then update the prediction for each sub-branch based on whether the change in orientation indicates that the instrument is heading or heading away from the corresponding sub-branch or whether the change in orientation indicates that the instrument is pointing or heading away from the corresponding sub-branch.
B.5. Selection of initial positioning
As described above, the
In some embodiments, the system may be configured to select the initial position of the instrument based on determining that the orientation of the instrument at the initial position is substantially aligned with the orientation of the current segment (e.g., aligned with a longitudinal axis of the current segment). As used herein, the system may consider the orientation of the instrument to be substantially aligned with the orientation of the current segment when the difference between the orientation of the instrument and the orientation of the current segment is less than a threshold difference. However, since the received positioning sensor data may not be registered to the model coordinate system, the system may not be able to directly compare the orientation of the instrument with the orientation of the current segment.
In one implementation, the system may be configured to receive an indication from a user that the instrument is aligned with the current segment. The user can confirm that the orientation of the instrument is currently aligned with the first segment based on other sensors of the system (e.g., a camera located at the distal end of the instrument). In one implementation, the system may provide instructions to the user to drive the instrument to a defined location (e.g., keel 310 of fig. 18) within the luminal network and retract the instrument at least a defined distance from the defined location. The system may determine the position of the distal end of the instrument at a location before or after retraction and set the orientation of the instrument at that point to an initial orientation that is substantially aligned with the orientation of the current segment.
In other implementations, the system may be configured or programmed to automatically select the initial position during driving of the instrument without receiving user input. In one embodiment, the system may track the orientation of the instrument as the instrument advances through the current segment over a period of time, and in response to the orientation of the instrument being substantially unchanged for a threshold period of time, the system may determine that the orientation of the instrument during the identified period is aligned with the orientation of the current segment. In some embodiments, the system may determine that the orientation of the instrument is substantially unchanged when the maximum difference between the orientations of the instrument measured over the period of time is less than a threshold difference.
B.6. Confirmation of registration process
In some embodiments, the system may be configured to perform a registration process in order to register the coordinate system of the localization sensor(s) to the coordinate system of the model of the luminal network. This registration may be stored in a registration data store 225 as shown in fig. 17. The registration process may include a process that facilitates determining a registration between the localized sensor coordinate system and the model coordinate system using the unregistered localization data. In some embodiments, the registration process may be performed based on: a history of data received from the positioning sensor(s) is maintained, and a shape formed from the positioning data history is matched to a candidate path along which an instrument based on the model of the anatomical structure can travel. During the registration process, the system may provide instructions to the user to drive the instrument along a predetermined registration path and track the position of the instrument based on the positioning sensor in response to the user driving the instrument along the registration path. For some procedures, the registration path may include a contralateral registration path that defines the shape of the path along which the instrument is to be driven for the registration process. The contralateral registration path may include: driving the instrument along a segment of the lumen network on an opposite side of its positioning relative to the target, retracting the instrument from the opposite side, and driving the instrument along a lateral side of the lumen network along a path to the target (also referred to as the target path). Thus, the contralateral registration path may include: driving the instrument along an opposite side branch of the luminal network that is outside of the target pathway (e.g., along a side branch of the luminal network that is opposite the target), returning the instrument to the target pathway, and driving the instrument along a side branch located along a portion of the target pathway.
In some procedures, the target may include a nodule (or lesion) to which the instrument may be driven to facilitate diagnosis and/or treatment. Thus, the memory can store the target path to the target in the model as well as the contralateral registration path. During the registration process, the system may be configured to confirm whether the user is currently driving the instrument to the correct branch of the luminal network defined by the contralateral registration path based on the prediction as to whether the instrument will advance along each of the sub-segments. Thus, the system may determine whether the instrument is positioned along the contralateral registration path based on the prediction. In one implementation, when approaching a bifurcation in the luminal network (e.g., a bifurcation near the main bronchus), the system can compare the probability that the instrument will advance into the bifurcation along the contralateral registration path to a threshold probability. When the probability is less than the threshold probability, the system may display an indication to the user that the user may not be driving toward the correct branch.
In another implementation, the system may determine that the instrument is advanced along the target path before being advanced along the contralateral registration path, which may indicate that the user inadvertently drives the instrument along a path that is not consistent with the contralateral registration path used in the registration process. Since the contralateral registration path may require the instrument to be driven along the contralateral path before being driven along the target path, the system may provide an indication that contralateral registration was unsuccessful in response to determining that the instrument was advanced along the target path before the instrument was advanced along the contralateral registration path.
The system may also be configured to display an indication of the positioning of the instrument relative to the model during a given procedure to provide feedback to the user. Accordingly, the system may determine the position of the distal end of the instrument relative to the model based on a plurality of data sources indicative of the location of the instrument. In certain implementations, the system may determine the position of the distal end of the instrument based on one or more of: positioning data received from a positioning sensor; a command set provided to control movement of the instrument, and prediction(s) regarding the instrument to advance into a sub-segment of the current segment. Thus, the prediction determined via the
C. Branch prediction based on registered positioning sensors.
After the system has performed the registration process (registering the localization sensor coordinate system to the model coordinate system), the system can use the data generated by the localization sensor(s) to determine an indication of the location of the distal end of the instrument with reference to the model associated with the model coordinate system. Using the registered data positioning data, the system may generate predictions about sub-segments of the luminal network into which the instrument is most likely to advance. Depending on the implementation, once the instrument has been advanced beyond a certain distance into the luminal network, the use of unregistered localized sensor data may not provide sufficient accuracy for branch prediction in the luminal network. For example, once the instrument has been advanced into the main bronchus, the system may not be able to perform branch prediction without registered positioning sensor data. Accordingly, various aspects of the present disclosure are also relevant to using the registered positioning data for branch prediction.
FIG. 20 illustrates an example skeleton-based model of a portion of a luminal network. In particular, skeleton-based
Fig. 21 is a flow diagram illustrating an example method operable by a robotic system or component(s) thereof for registration-based positioning sensor branch prediction, in accordance with various aspects of the present disclosure. For example, the steps of
At
In some implementations, when the system determines the orientation of the instrument at
At
At
According to an embodiment, the predicting may include: the identification of the sub-segment to which the instrument is most likely to advance, the probability that the instrument of each field in the sub-segment will advance into the corresponding sub-segment, an ordered list of sub-segments from highest probability to lowest probability, and so on. Thus, in certain embodiments, the prediction may include data indicative of a probability that the instrument will advance to each of the sub-segments. In some implementations, the system may assign a higher probability to sub-segments that have a smaller difference in orientation from the orientation of the instrument. Accordingly, the system may determine data indicative of a difference between the orientation of the instrument and the orientation of each of the sub-segments to help determine the corresponding probabilities.
In certain implementations, the system may also determine an angle between the orientation of the instrument and the orientation of each of the sub-segments. The angle between the orientation of a given sub-segment and the instrument may indicate a difference between the orientations. Thus, the system may be configured to assign a probability to a given sub-segment that is inversely proportional to the angle between the given sub-segment and the instrument. In one implementation, the system may determine the angle based on a dot product between the orientation of the instrument and the orientation of a given sub-segment. Since smaller angles may indicate greater alignment between a given sub-segment and the orientation of the instrument, the system may also use the inverse of the determined angle (invert) to calculate the probability of the given sub-segment.
This prediction may be used by the system as a data source for a fusion technique (e.g., the localization system 90 of fig. 15) for determining the localization of the distal end of the instrument relative to the model. In certain implementations, the system may determine the position of the distal end of the instrument based on one or more of: positioning data received from the positioning sensor(s), a set of commands provided to control movement of the instrument, and prediction(s) about the instrument to advance into each of the sub-segments.
In some implementations, in addition to the orientation-based prediction described above, the system may also apply auxiliary techniques for branch prediction. In one implementation, the assistance technique may include a location-based prediction that compares the location of the distal end of the instrument with the location of the beginning of each of the sub-segments. The system may determine an auxiliary prediction that the instrument will advance into each of the sub-segments based on the location of the distal end of the instrument. The prediction may be based on a location of the distal end of the instrument relative to each of the sub-segments. Further details and examples of location-based prediction techniques that may be used as an assistance technique are described in the above-referenced U.S. patent publication No. 2017/0084027. In one implementation, when the positioning sensor data indicates that the distal end of the instrument is positioned closer to one sub-segment than another sub-segment, the system may assign a higher probability to the closer sub-segment than the more distant sub-segment. For example, referring back to fig. 20, the system may assign a higher probability to a first one of the sub-segments 515 when the distal end of the
One advantage associated with orientation-based positioning sensor branch prediction over location-based branch prediction is that orientation-based prediction may be performed continuously during driving of the instrument through the luminal network. For example, location-based branch prediction techniques may not provide accurate predictions unless the distal end of the instrument is within a threshold distance of an entering sub-segment of the bifurcation of the current segment. Since the location-based prediction techniques rely on location sensor data, the location-based prediction techniques may be susceptible to errors in location sensor registration and jitter in the location sensor data. Thus, when the instruments are relatively far from the sub-segments, the distance between each of the sub-segments and the instrument may not indicate the sub-segment toward which the user is driving the instrument. In contrast, the orientation of the instrument may be more strongly correlated with the sub-segment into which the user is driving the instrument, even when the instrument is relatively far away from the bifurcation defined by the sub-segment. Thus, in some embodiments, the orientation-based branch prediction techniques described herein may be applied as the distal end of the instrument progresses from the beginning of the current segment to the end of the current segment. In other embodiments, the system may apply orientation-based branch prediction techniques regardless of the position of the distal end of the instrument within the current segment.
3. Implementation systems and terminology.
Implementations disclosed herein provide systems, methods, and apparatus for positioning sensor based branch prediction.
It should be noted that the terms "coupled," "coupled," or other variations of the word coupled as used herein may indicate either an indirect connection or a direct connection. For example, if a first element is "coupled" to a second element, the first element can be indirectly connected to the second element via another element or directly connected to the second element.
The functions described herein may be stored as one or more instructions on a processor-readable medium or a computer-readable medium. The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such media can include Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, compact disc read only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that computer-readable media may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code or data capable of being executed by a computing device or processor.
The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for proper operation of the method being described, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.
As used herein, the term "plurality" means two or more. For example, a plurality of components indicates two or more components. The term "determining" encompasses a wide variety of actions and, thus, "determining" can include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Further, "determining" may also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Further, "determining" may include resolving, selecting, choosing, establishing, and the like.
The phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" describes both "based only on" and "based at least on".
The previous description of the disclosed implementations is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other implementations without departing from the scope of the invention. For example, it should be understood that one of ordinary skill in the art will be able to employ a number of corresponding alternative and equivalent structural details, such as equivalent ways of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for generating specific actuation motions, and equivalent mechanisms for delivering electrical energy. Thus, the present invention is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.