阅读说明：本技术 一种人机协同力反馈脑室穿刺机器人装置 (Human-computer cooperative force feedback ventricular puncture robot device ) 是由 王君臣 杨晓涵 于 2021-08-23 设计创作，主要内容包括：一种人机协同力反馈脑室穿刺机器人装置,所述人机协同力反馈脑室穿刺机器人装置包括手术机器人系统(01),手术监测系统(02),手术控制系统(03),手术操作装置(04)和手术显示装置(05)；其中手术机器人系统(01)配置为执行穿刺手术动作；手术监测系统(02)配置为术中器械追踪和虚拟现实可视化导航；手术控制系统(03)配置为辅助穿刺操作；手术操作装置(04)为操作者手持的交互装置；手术显示装置(05)为外接的屏幕。(A human-computer cooperative force feedback ventricular puncture robot device comprises a surgical robot system (01), a surgical monitoring system (02), a surgical control system (03), a surgical operation device (04) and a surgical display device (05); wherein the surgical robotic system (01) is configured to perform a stab surgical action; the surgical monitoring system (02) is configured for intraoperative instrument tracking and virtual reality visualization navigation; the surgical control system (03) is configured to assist in the puncturing operation; the operation operating device (04) is an interactive device held by an operator; the operation display device (05) is an external screen.)

1. A human-computer cooperative force feedback ventricular puncture robot device is characterized by comprising a surgical robot system (01), a surgical monitoring system (02), a surgical control system (03), a surgical operation device (04) and a surgical display device (05); wherein the surgical robotic system (01) is configured to perform a stab surgical action; the surgical monitoring system (02) is configured for intraoperative instrument tracking and virtual reality visualization navigation; the surgical control system (03) is configured to assist in the puncturing operation; the operation operating device (04) is an interactive device held by an operator; the operation display device (05) is an external screen.

2. The human-computer cooperative force feedback ventricular puncture robot device according to claim 1, wherein the surgical robot system (01) comprises a 7-degree-of-freedom serial mechanical arm, a puncture surgical end effector and a matching control cabinet;

preferably, the surgical monitoring system (02) comprises a visual servo navigation system and a force feedback system; the force feedback system comprises a pressure sensor of the end effector of the puncture surgery, a force sense interaction device and an upper computer; the visual servo navigation system comprises a binocular camera and a visual marker fixedly connected to the puncture operation end effector and the operation object;

preferably, the operation control system (03) is a computer, and the computer comprises an image processing module, a force sense signal processing module, an interactive device communication module and a visual navigation display module;

preferably, the surgical operation device (04) is configured to map the hand motion into the puncture motion of the surgical robot system (03) by operating the interactive device.

3. The human-computer cooperative force feedback ventricular puncture robot device according to claim 1 or 2, wherein the surgical robot system (01) comprises a 7-degree-of-freedom serial mechanical arm (06), a puncture surgical end effector (07) and a mating control cabinet (08);

preferably, the puncture surgical end effector (07) comprises a puncture drill bit (09), a puncture axial force measuring module (10) and a feeding mechanism (11);

preferably, the piercing drill (09) comprises a twist drill (12), a bearing seat (13), a coupler (14), a motor support (15) and a motor (16), wherein the bearing seat (13) supports the twist drill (12), the motor support (15) supports the motor (16), and the coupler (14) is connected with the motor (16) and the twist drill (12) and transmits the power of the motor (16) to the twist drill (12);

preferably, the puncture axial force measuring module (10) comprises a pre-tightening force module (17), a movable force sensor support (18), a pressure sensor (19), a drill bit connecting block (20), a pressure sensor contact block (21), a fixed force sensor support (22), a sliding block (23), a guide rail (24) and a bottom plate (25); the pre-tightening force module (17), the movable force sensor support (18), the guide rail (24) and the fixed force sensor support (22) are sequentially fixed on the bottom plate (25); the pretightening force module (17) is provided with a threaded hole facing the movable force sensor bracket (18), and a pressure sensor (19) is fixed on the movable force sensor bracket (18) and the fixed force sensor bracket (22); the slide block (23) is arranged on the guide rail (24), preferably, the slide block (23) adopts a linear bearing and can freely slide on the guide rail; a pressure sensor contact block (21) and a drill bit connecting block (20) are sequentially and fixedly connected to the sliding block (23); the drill bit connecting block (20) is fixedly connected with the puncture drill bit (09) and transmits the axial force borne by the puncture drill bit (09) to the puncture axial force measuring module (10), in particular to the pressure sensor contact block (21) fixedly connected with the puncture axial force measuring module; the surface of the pressure sensor contact block (21) in contact with the pressure sensor (19) is a plane;

preferably, the feeding mechanism (11) comprises a direct current motor (26), a synchronous belt mechanism (27), a lead screw nut mechanism (28), a slide block (29), a photoelectric switch (30) and a mechanical arm connecting piece (31); wherein, the synchronous belt mechanism (27) converts the rotation output by the direct current motor (26) into the input of the screw nut mechanism (28); the screw rod nut mechanism (28) converts the input rotation into linear motion and drives the slide block (29) to do linear motion; the sliding block (29) is fixedly connected with a metal sheet and is used for limiting in cooperation with the photoelectric switch (30); the sliding block (29) is fixedly connected with the puncture axial force measuring module (10); preferably, the photoelectric switch (30) plays a role in limiting the sliding block (29) by detecting the blocking of a metal sheet fixedly connected with the sliding block (29) to the photoelectric switch light path; the mechanical arm connecting piece (31) is connected with the puncture operation end effector (07) and the 7-degree-of-freedom mechanical arm (06).

4. The human-computer-assisted force feedback ventricular puncture robotic device according to any one of claims 1-3, wherein the surgical monitoring system (02) is configured to implement visual servoing and visual navigation; the operation monitoring system (02) comprises a visual servo navigation system and a force feedback system, wherein the force feedback system comprises a pressure sensor for puncturing an operation end effector, a force sense interaction device and an upper computer; the visual servo-navigation system includes a binocular camera and a visual marker attached to the penetrating surgical end effector and the surgical object.

5. The human-computer cooperative force feedback ventricular puncture robot apparatus according to any one of claims 1-4, wherein the surgery control system (03) is configured to control the whole process of the ventricular puncture surgery;

preferably, the operation control system (03) comprises a computer, and the computer comprises an image processing module, a motor control module, a force sense signal processing module, an interaction device communication module and a visual navigation display module.

6. The control method of the human-computer cooperation force feedback ventricular puncture robot device according to any one of claims 1-5, wherein the control method comprises preoperative planning, visual navigation and human-computer cooperation.

7. The control method of claim 6, wherein the preoperative planning comprises: obtaining image information of a surgical object; image segmentation; determining a surgical target; registering the surgical object; in addition to this, there is a need to prepare the surgical environment, including: fixedly connecting a visual Marker (Marker) on the operation object; ensuring that the surgical object is within the field of view of the binocular camera and is fixed; the surgical robot is brought into a working position, i.e. a position that can be seen by the binocular camera.

8. The control method according to claim 6 or 7, wherein the visual navigation and man-machine coordination comprises:

firstly, enabling a puncture operation end effector to reach a puncture starting pose under visual guidance by a robot according to a preoperative planned track; the method comprises the steps of detecting the corner points of images of a binocular camera in real time, calculating the position of a puncture operation end effector fixedly connected with a visual marker (marker) in real time based on the rapid template matching of the visual marker (marker), and displaying the operation state in real time in host computer software by combining the pose of a mechanical arm;

after the robot reaches the starting position of puncture, an operator operates the interaction equipment to carry out man-machine cooperative operation; obtaining pressure sensor initial data F after reaching the puncture initial point₁₀，F₂₀(ii) a Then, real-time reading of the pressure sensor data is started and the formula F ═ F (F) is used₂₁-F₂₀)-(F₁₁-F₁₀) (2-3) calculating the puncture force; the puncture force can be proportionally fed back to the hand of an operator through the interactive device; the operation of an operator in the puncture process has dual guarantees of operation navigation visual display and interactive equipment force feedback;

an operator firstly controls the drill to feed through the interaction equipment until a target position is reached, and the drill does not rotate at the moment; then the operator pulls the trigger bit of the interaction equipment to rotate; the operator reasonably controls the feeding rate according to the force fed back from the hand; when the drill bit breaks through the skull, the rotation and the feeding of the drill bit are immediately and automatically stopped according to the sudden change of the puncture force; then the operator performs other operations;

the robot is used for puncture surgery, and a vision servo navigation system is required to provide target pose information of a serial mechanical arm with 7 degrees of freedom; the method comprises the steps that a target pose of a TCP coordinate system in an image coordinate system can be given in a preoperative planning stage, the relative pose of the TCP coordinate system of the 7-freedom-degree serial mechanical arm relative to a skull visual Marker (Marker) local coordinate system in an actual operation space is obtained through image registration mapping, and the 7-freedom-degree serial mechanical arm is further controlled to move to a correct puncture pose; the detailed coordinate system transformation relationship is shown in fig. 8;

wherein, { O_TCPThe coordinate system of the surgical tool is defined as the coordinate system,coordinate System of visual Marker (Marker) for fixation to the end of a 7-DOF tandem robot arm, { O_CThe coordinate system of the camera is used as the coordinate system,for a visual Marker (Marker) coordinate system attached to the target bone, { O_MIs CT image space coordinate system and O_VTCPThe 7-degree-of-freedom serial mechanical arm is connected with a surgical tool coordinate system when the mechanical arm reaches a target position; the transformation relationship between the above coordinate systems is as follows:

the pose transformation matrix of a 7-degree-of-freedom serial mechanical arm terminal visual Marker (Marker) coordinate system relative to an operation tool coordinate system is obtained by hand-eye calibration;the pose transformation matrix is a pose transformation matrix of a coordinate system of a visual Marker (Marker) at the tail end of a 7-degree-of-freedom serial mechanical arm relative to a camera coordinate system;is a pose transformation matrix of a visual Marker (Marker) coordinate system fixedly connected with a target bone relative to a camera coordinate system;the pose transformation matrix of a local coordinate system of a CT image coordinate system fixedly connected with a visual Marker (Marker) relative to a target bone can be obtained through image registration;the pose transformation matrix of the target surgical tool coordinate system relative to the image coordinate system is given in the preoperative planning stage; according to the chain rule, the coordinate transformation matrix of the 7-degree-of-freedom tandem manipulator TCP from the starting position to the target position can be obtained by the following formula:

t in the formula (3-1) is a transformation matrix of a TCP coordinate system at the tail end of the 7-freedom-degree serial mechanical arm, the transformation matrix is sent to a 7-freedom-degree serial mechanical arm controller to control the 7-freedom-degree serial mechanical arm to move, and the end effector of the puncture operation is brought to a target posture from an initial position so as to start the puncture operation;

the visual servo uses a target positioning and operation navigation method based on binocular vision, and comprises the identification and positioning of an X angular point, the registration and identification of a visual Marker (Marker), the registration and positioning of the tip of a surgical instrument and the image registration; the algorithm is accelerated by combining with a GPU, and for 960 × 480 images, more than 100 frames can be processed every second, so that the requirement of the system on real-time performance is met; the surgical instruments are quickly identified and positioned, the model of the brain and the surgical instrument model in the early stage are led into a virtual scene, and the visual navigation of the operation can be realized through image registration.

Technical Field

The invention relates to the field of neurosurgical operation robots, in particular to a human-computer cooperative force feedback ventricular puncture robot device and a control method thereof.

Background

Neurosurgery, or the operation of making an incision in the skull for the purpose of relieving pain, etc., has been known for thousands of years. According to archaeological findings and literature, imprintants have mastered the technique of disease treatment by drilling holes in the skull of a person, as early as in the stoneware era. Al-Zahrawi physicians have performed neurosurgical treatments of craniocerebral injuries, skull fractures, hydrocephalus and subdural effusion in the middle and old European century. However, neurosurgery has been rapidly developed in nearly one hundred years due to limitations such as scientific and technological development. The coverage area of the existing neurosurgery relates to the treatment of meningitis and other central system infection diseases, hydrocephalus, skull fracture and other head trauma, spinal cord and peripheral nerve tumors, cerebral hemorrhage, vascular malformation, dyskinesia diseases, brain cancers and other diseases.

The brain is the only organ completely encapsulated in the bone, and has functional uniqueness compared with other organs such as the kidney and the liver, namely, the function of each part of the brain can not be replaced by tissues at other positions. Thus, the mortality rate and disability rate are high once head injury occurs.

The most important and effective means for saving the life of craniocerebral trauma patients internationally recognized clinically is to carry out timely and accurate ventricular puncture drainage operation on the patients. Ventricular puncture Drainage (EVD) is a neurosurgical procedure that reduces intracranial pressure by externally opening access to the Ventricular area, a first aid for patients with cerebral trauma. In the case of serious intracranial hypertension caused by hydrocephalus, when the condition is critical and cerebral hernia or coma occurs, ventricular puncture and drainage should be used as emergency pressure-reducing rescue measures, so as to create conditions for further diagnosis and treatment. For patients with hemorrhage in ventricle, puncturing and draining bloody cerebrospinal fluid can relieve ventricular reaction and prevent ventricular system obstruction.

Ventricular puncture drainage surgery is one of the most difficult surgical procedures in trauma control surgery in clinical practice. Before the operation, a doctor needs to read the CT image to determine the operation position, the positioning accuracy is high in the operation, and the professional and psychological quality requirements of the operating doctor are high. However, in the case of shortage of professionals in the emergency environment, there is a need to realize precision, visualization and automation of craniocerebral puncture drainage operation.

Aiming at the requirement, the invention provides a technical route of a human-computer cooperative force feedback ventricular puncture robot device. The robot puncture skull completes the precise, visual and automatic ventricle puncture operation under the conditions of preoperative operation planning, intraoperative operation navigation and man-machine cooperation.

The human-computer cooperative force feedback ventricular puncture robot device provided by the invention has the advantages that the force feedback is combined with the visual navigation, so that an operator can remotely operate the operation through the robot, and the requirements on the professional and psychological quality of the operator are reduced. The robot operation and the human in-loop combination ensure the safety of the operation. The operator can control the operation process through two aspects of force feedback and visual navigation. Realizes the human-in-the-loop and ventricle puncture operation combined with visual force sense and provides a thought for the neurosurgery operations of the same type.

The invention provides a hardware system and a control method for realizing automatic ventricular puncture surgery.

Disclosure of Invention

The embodiment of the invention provides a human-computer cooperative force feedback ventricular puncture robot device, which comprises a surgical robot system (01), a surgical monitoring system (02), a surgical control system (03), a surgical operation device (04) and a surgical display device (05); wherein the surgical robotic system (01) is configured to perform a stab surgical action; the surgical monitoring system (02) is configured for intraoperative instrument tracking and virtual reality visualization navigation; the surgical control system (03) is configured to assist in the puncturing operation; the operation operating device (04) is an interactive device held by an operator; the operation display device (05) is an external screen.

According to one embodiment of the present invention, for example, a surgical robotic system (01) includes a 7-degree-of-freedom tandem robotic arm, a piercing surgical end effector, and a mating control cabinet;

preferably, the surgical monitoring system (02) comprises a visual servo navigation system and a force feedback system; the force feedback system comprises a pressure sensor of the end effector of the puncture surgery, a force sense interaction device and an upper computer; the visual servo navigation system comprises a binocular camera and a visual marker fixedly connected to the puncture operation end effector and the operation object;

preferably, the operation control system (03) is a computer, and the computer comprises an image processing module, a force sense signal processing module, an interactive device communication module and a visual navigation display module;

preferably, the surgical operation device (04) is configured to map the hand motion into the puncture motion of the surgical robot system (03) by operating the interactive device.

According to one embodiment of the invention, for example, a surgical robotic system (01) includes a 7-degree-of-freedom tandem robotic arm (06), a piercing surgical end effector (07), and a mating control cabinet (08);

preferably, the puncture surgical end effector (07) comprises a puncture drill bit (09), a puncture axial force measuring module (10) and a feeding mechanism (11);

preferably, the piercing drill (09) comprises a twist drill (12), a bearing seat (13), a coupler (14), a motor support (15) and a motor (16), wherein the bearing seat (13) supports the twist drill (12), the motor support (15) supports the motor (16), and the coupler (14) is connected with the motor (16) and the twist drill (12) and transmits the power of the motor (16) to the twist drill (12);

preferably, the puncture axial force measuring module (10) comprises a pre-tightening force module (17), a movable force sensor support (18), a pressure sensor (19), a drill bit connecting block (20), a pressure sensor contact block (21), a fixed force sensor support (22), a sliding block (23), a guide rail (24) and a bottom plate (25); the pre-tightening force module (17), the movable force sensor support (18), the guide rail (24) and the fixed force sensor support (22) are sequentially fixed on the bottom plate (25); the pretightening force module (17) is provided with a threaded hole facing the movable force sensor bracket (18), and a pressure sensor (19) is fixed on the movable force sensor bracket (18) and the fixed force sensor bracket (22); the slide block (23) is arranged on the guide rail (24), preferably, the slide block (23) adopts a linear bearing and can freely slide on the guide rail; a pressure sensor contact block (21) and a drill bit connecting block (20) are sequentially and fixedly connected to the sliding block (23); the drill bit connecting block (20) is fixedly connected with the puncture drill bit (09) and transmits the axial force borne by the puncture drill bit (09) to the puncture axial force measuring module (10), in particular to the pressure sensor contact block (21) fixedly connected with the puncture axial force measuring module; the surface of the pressure sensor contact block (21) in contact with the pressure sensor (19) is a plane;

preferably, the feeding mechanism (11) comprises a direct current motor (26), a synchronous belt mechanism (27), a lead screw nut mechanism (28), a slide block (29), a photoelectric switch (30) and a mechanical arm connecting piece (31); wherein, the synchronous belt mechanism (27) converts the rotation output by the direct current motor (26) into the input of the screw nut mechanism (28); the screw rod nut mechanism (28) converts the input rotation into linear motion and drives the slide block (29) to do linear motion; the sliding block (29) is fixedly connected with a metal sheet and is used for limiting in cooperation with the photoelectric switch (30); the sliding block (29) is fixedly connected with the puncture axial force measuring module (10); preferably, the photoelectric switch (30) plays a role in limiting the sliding block (29) by detecting the blocking of a metal sheet fixedly connected with the sliding block (29) to the photoelectric switch light path; the mechanical arm connecting piece (31) is connected with the puncture operation end effector (07) and the 7-degree-of-freedom mechanical arm (06).

According to one embodiment of the invention, for example, a surgical monitoring system (02) is configured to implement visual servoing and visual navigation; the operation monitoring system (02) comprises a visual servo navigation system and a force feedback system, wherein the force feedback system comprises a pressure sensor for puncturing an operation end effector, a force sense interaction device and an upper computer; the visual servo-navigation system includes a binocular camera and a visual marker attached to the penetrating surgical end effector and the surgical object.

According to one embodiment of the invention, for example, the surgical control system (03) is configured to control the overall process of a ventricular puncture procedure;

preferably, the operation control system (03) comprises a computer, and the computer comprises an image processing module, a motor control module, a force sense signal processing module, an interaction device communication module and a visual navigation display module.

The embodiment of the invention also provides a control method of the human-computer cooperation force feedback ventricular puncture robot device, which comprises preoperative planning, visual navigation and human-computer cooperation.

According to one embodiment of the present invention, for example, the preoperative planning includes: obtaining image information of a surgical object; image segmentation; determining a surgical target; registering the surgical object; in addition to this, there is a need to prepare the surgical environment, including: fixedly connecting a visual Marker (Marker) on the operation object; ensuring that the surgical object is within the field of view of the binocular camera and is fixed; the surgical robot is brought into a working position, i.e. a position that can be seen by the binocular camera.

According to one embodiment of the present invention, for example, visual navigation and human-machine collaboration comprises:

firstly, enabling a puncture operation end effector to reach a puncture starting pose under visual guidance by a robot according to a preoperative planned track; the method comprises the steps of detecting the corner points of images of a binocular camera in real time, calculating the position of a puncture operation end effector fixedly connected with a visual marker (marker) in real time based on the rapid template matching of the visual marker (marker), and displaying the operation state in real time in host computer software by combining the pose of a mechanical arm;

after the robot reaches the starting position of puncture, an operator operates the interaction equipment to carry out man-machine cooperative operation; obtaining pressure sensor initial data F after reaching the puncture initial point₁₀，F₂₀(ii) a Then, real-time reading of the pressure sensor data is started and the formula F ═ F (F) is used₂₁-F₂₀)-(F₁₁-F₁₀) (2-3) calculating the puncture force; the puncture force can be proportionally fed back to the hand of an operator through the interactive device; the operation of an operator in the puncture process has dual guarantees of operation navigation visual display and interactive equipment force feedback;

an operator firstly controls the drill to feed through the interaction equipment until a target position is reached, and the drill does not rotate at the moment; then the operator pulls the trigger bit of the interaction equipment to rotate; the operator reasonably controls the feeding rate according to the force fed back from the hand; when the drill bit breaks through the skull, the rotation and the feeding of the drill bit are immediately and automatically stopped according to the sudden change of the puncture force; then the operator performs other operations;

the robot is used for puncture surgery, and a vision servo navigation system is required to provide target pose information of a serial mechanical arm with 7 degrees of freedom; the method comprises the steps that a target pose of a TCP coordinate system in an image coordinate system can be given in a preoperative planning stage, the relative pose of the TCP coordinate system of the 7-freedom-degree serial mechanical arm relative to a skull visual Marker (Marker) local coordinate system in an actual operation space is obtained through image registration mapping, and the 7-freedom-degree serial mechanical arm is further controlled to move to a correct puncture pose; the detailed coordinate system transformation relationship is shown in fig. 8;

wherein, { O_TCPThe coordinate system of the surgical tool is defined as the coordinate system,coordinate System of visual Marker (Marker) for fixation to the end of a 7-DOF tandem robot arm, { O_CThe coordinate system of the camera is used as the coordinate system,for a visual Marker (Marker) coordinate system attached to the target bone, { O_MIs CT image space coordinate system and O_VTCPThe 7-degree-of-freedom serial mechanical arm is connected with a surgical tool coordinate system when the mechanical arm reaches a target position; the transformation relationship between the above coordinate systems is as follows:

the pose transformation matrix of a 7-degree-of-freedom serial mechanical arm terminal visual Marker (Marker) coordinate system relative to an operation tool coordinate system is obtained by hand-eye calibration;the pose transformation matrix is a pose transformation matrix of a coordinate system of a visual Marker (Marker) at the tail end of a 7-degree-of-freedom serial mechanical arm relative to a camera coordinate system;is a pose transformation matrix of a visual Marker (Marker) coordinate system fixedly connected with a target bone relative to a camera coordinate system;the pose transformation matrix of a local coordinate system of a CT image coordinate system fixedly connected with a visual Marker (Marker) relative to a target bone can be obtained through image registration;the pose transformation matrix of the target surgical tool coordinate system relative to the image coordinate system is given in the preoperative planning stage; according to the chain rule, the coordinate transformation matrix of the 7-degree-of-freedom tandem manipulator TCP from the starting position to the target position can be obtained by the following formula:

t in the formula (3-1) is a transformation matrix of a TCP coordinate system at the tail end of the 7-freedom-degree serial mechanical arm, the transformation matrix is sent to a 7-freedom-degree serial mechanical arm controller to control the 7-freedom-degree serial mechanical arm to move, and the end effector of the puncture operation is brought to a target posture from an initial position so as to start the puncture operation;

the visual servo uses a target positioning and operation navigation method based on binocular vision, and comprises the identification and positioning of an X angular point, the registration and identification of a visual Marker (Marker), the registration and positioning of the tip of a surgical instrument and the image registration; the algorithm is accelerated by combining with a GPU, and for 960 × 480 images, more than 100 frames can be processed every second, so that the requirement of the system on real-time performance is met; the surgical instruments are quickly identified and positioned, the model of the brain and the surgical instrument model in the early stage are led into a virtual scene, and the visual navigation of the operation can be realized through image registration.

The invention has the following beneficial technical effects:

(1) the human-computer cooperative force feedback ventricular puncture robot device provided by the invention has complete automatic control hardware configuration and adaptive mechanical configuration, so that the realization of precision, visualization and automation of the craniocerebral puncture drainage operation becomes possible.

(2) The robot puncture skull completes the precise, visual and automatic ventricle puncture operation under the conditions of preoperative operation planning, intraoperative operation navigation and man-machine cooperation.

(3) The human-computer cooperative force feedback ventricular puncture robot device provided by the invention has the advantages that the force feedback is combined with the visual navigation, so that an operator can remotely operate the operation through the robot, and the requirements on the professional and psychological quality of the operator are reduced. The robot operation and the human in-loop combination ensure the safety of the operation. The operator can control the operation process through two aspects of force feedback and visual navigation. Realizes the human-in-the-loop and ventricle puncture operation combined with visual force sense and provides a thought for the neurosurgery operations of the same type.

(4) The invention adopts a master-slave control mode, can realize remote operation, and has strategic and economic significance to remote areas, such as islands, frontier regions and the like.

Drawings

FIG. 1 is a schematic structural diagram of a human-computer cooperative force feedback ventricular puncture robot device provided in an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a surgical robot system 01 in a human-computer cooperative force feedback ventricular puncture robot device provided in an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a puncture surgical end effector in a human-computer cooperative force feedback ventricular puncture robot device provided in an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a puncture drill 09 in the human-computer cooperative force feedback ventricular puncture robot device provided by the embodiment of the invention;

fig. 5 is a schematic structural diagram of a puncture axial force measuring module 10 in a human-computer cooperative force feedback ventricular puncture robot device provided in an embodiment of the present invention;

FIG. 6 is a schematic view of a puncture axial force measurement module force analysis;

fig. 7 is a schematic structural diagram of a feeding mechanism 11 in a human-computer cooperative force feedback ventricular puncture robot device provided by an embodiment of the invention;

fig. 8 is a coordinate system transformation relation diagram of the human-computer cooperative force feedback ventricular puncture robot device according to the embodiment of the present invention.

Fig. 9 is a flowchart of a control algorithm of the human-computer cooperative force feedback ventricular puncture robot device according to the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings. Those skilled in the art will appreciate that the present invention is not limited to the drawings and the following examples.

The application discloses human-computer cooperation force feedback ventricles of brain puncture robot device includes operation robot system, operation monitoring system, operation control system, operation operating means and operation display device. Wherein the surgical robotic system is configured to perform a puncture surgical procedure; the operation monitoring system is configured to track instruments in operation and visually navigate virtual reality; the surgical control system is configured to assist in the puncturing operation; the operation device is an interactive device held by an operator; the operation display device is an external screen.

The technical details of the surgical robotic system, the surgical monitoring system, and the surgical control system will be described in detail with reference to the embodiments.

Example 1 Assembly of the robot System for ventricular puncture surgery

As shown in fig. 1, the human-computer cooperative force feedback ventricular puncture robot device includes a surgical robot system 01, a surgical monitoring system 02, a surgical control system 03, a surgical operation device 04, and a surgical display device 05.

The surgical robot system 01, also called a puncture surgery execution system, includes a 7-degree-of-freedom serial mechanical arm, a puncture surgery end effector, and a matching control cabinet.

The surgical monitoring system 02 is configured to monitor the progress of the surgery and provide feedback information to the surgeon. The surgical monitoring system 02 includes a visual servo-navigation system and a force feedback system. The force feedback system comprises a pressure sensor of the puncture operation end effector, a force sense interaction device and an upper computer. The visual servo navigation system includes a binocular camera and a visual marker (marker) attached to the penetrating surgical end effector and the surgical object.

The operation control system 03 can be, for example, a computer including an image processing module, a force sense signal processing module, an interactive device communication module, and a visual navigation display module, and the operation control system 03 is configured to provide a flow of a human-computer collaborative ventricular puncture operation control algorithm.

The operation device 04 is an interactive device actually used by a doctor in a man-machine cooperation puncture operation process; the surgical manipulation device 04 is configured to map the hand motion into the puncture motion of the surgical robotic system 03 by manipulating the interactive apparatus.

The operation display device 05 is a display externally connected to the operation control system 03 and used for displaying the operation process.

In the process of the man-machine cooperation ventricular puncture surgery, a 7-degree-of-freedom serial mechanical arm and a puncture surgery end effector in a surgical robot system automatically track to a preset puncture position under the guidance of a visual servo navigation system, and then man-machine cooperation puncture is carried out.

The operator controls the interaction device to perform the puncturing operation. The actions of the interactive equipment are mapped to the puncture actuator in proportion through the upper computer and the control cabinet, so that the feeding amount of the drill bit is controlled. The trigger of the interaction device controls the drill bit rotation. When the tail end of the actuator feels the puncture axial force, the axial force is fed back to the interaction equipment in proportion through the control cabinet and the upper computer, and the interaction equipment applies feedback force to the hand of an operator. Thereby the operator can grasp the puncturing force.

Example 2 composition of parts of robot System for ventricular puncture surgery

1. Surgical robot system

Fig. 2 is a diagram illustrating a structure of a surgical robot system 01 in a human-computer cooperative force feedback ventricular puncture robot apparatus according to an embodiment of the present invention. As shown in fig. 2, the surgical robotic system 01 includes a 7-degree-of-freedom serial mechanical arm 06, a puncture surgical end effector 07, and a mating control cabinet 08.

Fig. 3 shows the structure of the puncture surgical end effector 07. The end effector 07 for the puncture surgery is designed with 2 degrees of freedom for the requirement of ventricular puncture, which are the rotation of the drill and the feeding of the drill, respectively. As shown in fig. 3, the piercing surgical end effector 07 includes three parts, a piercing drill 09, a piercing axial force measuring module 10, and a feeding mechanism 11. The structure of each portion is illustrated below.

1.1. Structure of piercing drill 09

Fig. 4 shows the structure of the piercing drill 09. As shown in fig. 4, the piercing drill 09 includes a twist drill 12, a bearing housing 13, a coupling 14, a motor bracket 15, and a motor 16. Wherein, bearing frame 13 provides the support for twist drill 12, and motor support 15 provides the support for motor 16, and shaft coupling 14 connects motor 16 and twist drill 12, gives twist drill 12 with motor 16's power transmission.

1.2. Structure of puncture axial force measuring module 10

In the ventricular puncture process, in order to ensure safety, the axial stress of the drill bit in the puncture process needs to be measured. The stress information has two purposes, and real force feedback is provided for an operator in the man-machine cooperation puncture process, so that the operator feels the state of the operation, and the safety can be ensured to a great extent; the second purpose is to achieve autonomous puncture under force control. The puncture force was measured by differential method. Fig. 5 shows the structure of the piercing axial force measuring module 10. As shown in fig. 5, the puncture axial force measurement module 10 includes a preload module 17, a movable force sensor support 18, a pressure sensor 19, a drill connection block 20, a pressure sensor contact block 21, a fixed force sensor support 22, a slider 23, a guide rail 24, and a base plate 25, as shown in fig. 5.

The pretightening force module 17, the movable force sensor support 18, the guide rail 24 and the fixed force sensor support 22 are sequentially fixed on the bottom plate 25. The pretensioning module 17 has a threaded hole facing the mobility sensor support 18. The movable force sensor support 18 and the fixed force sensor support 22 are fixed with a pressure sensor 19. The guide rail 24 is provided with a slider 23. The slider 23 may be, for example, a linear bearing and may freely slide on a guide rail. And a pressure sensor contact block 21 and a drill bit connecting block 20 are sequentially and fixedly connected to the sliding block 23. The drill bit connecting block 20 is fixedly connected with the puncture drill bit 09 and transmits the axial force applied to the puncture drill bit to the puncture axial force measuring module 10, in particular to the pressure sensor contact block 21 fixedly connected with the puncture axial force measuring module. The processing of the pressure sensor contact block 21 is required to ensure that the surface contacting the pressure sensor 19 in fig. 5 is a plane surface, so as to ensure the accuracy of the measurement of the pressure sensor.

The design requirement of the puncture axial force measuring module 10 is to reduce the weight as much as possible under the condition of ensuring the measurement of the puncture axial force, so as to facilitate the control of the serial mechanical arm. Therefore, it is preferable to design the weight-reducing grooves at the time of design and select aluminum as the working material.

The installation process of the puncture axial force measurement module 10 is as follows: the mobility sensor support 18 is not screwed down during initial installation so that it can slide in the kidney hole shown in fig. 5, with the rest of the assembly in turn. The pretension module 17 is then screwed in. The screws are pressed against the movable force sensor support 18, so that the position of the movable force sensor support is fixed and 2 pressure sensors have pretightening force. At this time, the screw of the movable force sensor holder 18 is tightened to fix the position thereof. The screws on the pretension module 17 are then removed.

After the installation, the puncture axial force measuring module 10 can be regarded as a guide rail sliding block model capable of slightly sliding, and two ends of the puncture axial force measuring module are provided with pressure sensors to be contacted with the sliding blocks. The mechanism can be analyzed using the force analysis diagram shown in fig. 6. F₁，F₂Respectively, the 2-terminal pressure sensor readings. G is the gravity of the whole of the sliding block and the drill bit fixedly connected on the sliding block, and the component force of the sliding block along the inclined plane direction is marked as G₁. F is the penetration force during drilling, and the value is 0 during non-drilling.

According to theoretical mechanics knowledge, a drill bit is fixedly connected to a sliding block, and the sliding block slightly moves relative to a sliding rail under the action of external force, so that the reading of pressure sensors at two ends changes. Therefore, the operating condition of the ventricular puncture robot before and after puncture can be regarded as the inclined plane slide block. The friction force of the sliding rail sliding block is far smaller than the gravity of the drill bit and the puncture force of the drilled bone, and the judgment of the tissue type being drilled in the bone drilling process cannot be influenced by the friction force, so that the friction force can be ignored in stress analysis.

During the operation, the robot firstly reaches the planned puncture position under the visual guidance, and the puncture is ready. The readings of the pressure sensors in this state are respectively denoted as F₁₀，F₂₀The mechanical formula along the direction of the inclined plane is:

F₁₀+0＝G₁+F₂₀(2-1) during the puncturing, since the attitude is not changed but only the drill is fed, G₁And is not changed. Mechanical formula along the direction of the inclined plane:

F₁₁+F＝G₁+F₂₁(2-2) formula (I) wherein F₁₁，F₂₁Is the real-time reading of the pressure sensor during the puncture process. The calculation formula of the real-time puncture force F obtained by sorting is as follows:

F＝(F₂₁-F₂₀)-(F₁₁-F₁₀) (2-3)

1.3. structure of feed mechanism 11

Fig. 7 shows the structure of the feed mechanism 11. As shown in fig. 7, the feeding mechanism 11 includes a dc motor 26, a timing belt mechanism 27, a lead screw nut mechanism 28, a slider 29, an opto-electric switch 30, and a robot arm link 31.

Wherein the timing belt mechanism 27 converts the rotation output by the dc motor 26 into an input to the lead screw nut mechanism 28. The lead screw nut mechanism 28 converts the input rotation into linear motion and drives the slider 29 into linear motion. The sliding block 29 is fixedly connected with a metal sheet and is used for limiting in cooperation with the photoelectric switch 30. The sliding block 29 is fixedly connected with the puncture axial force measuring module 10. The photoelectric switch 30 plays a role in limiting the sliding block 29 by detecting the blocking of the optical path of the photoelectric switch by the metal sheet fixedly connected with the sliding block 29. The robot arm link 31 is used to connect the puncture surgical end effector 07 to the 7-degree-of-freedom robot arm 06.

2. Operation monitoring system

The surgical monitoring system 02 is configured to implement visual servoing and visual navigation. The operation monitoring system 02 comprises a visual servo navigation system and a force feedback system, wherein the force feedback system comprises a pressure sensor for puncturing an operation end effector, a force sense interaction device and an upper computer; the visual servo navigation system includes a binocular camera and a visual marker (marker) attached to the penetrating surgical end effector and the surgical object.

When a robot is used for puncture surgery, a vision servo navigation system is required to provide target pose information of a 7-degree-of-freedom serial mechanical arm (because the tail end of the 7-degree-of-freedom serial mechanical arm is fixedly connected with a puncture actuator, the tail end pose of the 7-degree-of-freedom serial mechanical arm and the pose of the puncture actuator are in one-to-one correspondence). The preoperative planning stage can provide a target pose of the TCP coordinate system in an image coordinate system, the relative pose of the TCP coordinate system of the 7-freedom-degree serial mechanical arm relative to a skull Marker local coordinate system in an actual operation space is obtained through image registration mapping, and the 7-freedom-degree serial mechanical arm is further controlled to move to a correct puncture pose. The detailed coordinate system transformation relationship is shown in fig. 8.

Wherein, { O_TCPThe coordinate system of the surgical tool is defined as the coordinate system,for a Marker coordinate system fixed to the end of a 7 degree-of-freedom serial robot arm, { O_CThe coordinate system of the camera is used as the coordinate system,(O) Marker coordinate System attached to the target bone_MIs CT image space coordinate system and O_VTCPAnd 7 degrees of freedom are connected with a surgical tool coordinate system when the mechanical arm reaches a target position in series. The transformation relationship between the above coordinate systems is as follows:

the pose transformation matrix of a Marker coordinate system at the tail end of a 7-degree-of-freedom serial mechanical arm relative to a surgical tool coordinate system is obtained by calibrating hands and eyes;the pose transformation matrix is a pose transformation matrix of a Marker coordinate system at the tail end of a 7-degree-of-freedom serial mechanical arm relative to a camera coordinate system;is a pose transformation matrix of a Marker coordinate system fixedly connected with a target bone relative to a camera coordinate system;the pose transformation matrix of a local coordinate system of a Marker fixedly connected with a target bone by a CT image coordinate system can be obtained by image registration;the pose transformation matrix of the target surgical tool coordinate system relative to the image coordinate system is provided in the preoperative planning stage. According to the chain rule, the coordinate transformation matrix of the 7-degree-of-freedom tandem manipulator TCP from the starting position to the target position can be obtained by the following formula:

in the formula (3-1), T is a transformation matrix of a TCP coordinate system at the tail end of the 7-freedom-degree serial mechanical arm, and the transformation matrix is sent to a controller of the 7-freedom-degree serial mechanical arm to control the motion of the 7-freedom-degree serial mechanical arm, so that the end effector of the puncture operation is brought to a target posture from an initial position, and then the puncture operation can be started.

The visual servo uses a target positioning and operation navigation method based on binocular vision, and comprises the identification and positioning of an X-corner point, the registration and identification of a Marker, the registration and positioning of the tip of a surgical instrument and the image registration. The algorithm is accelerated by combining with the GPU, and for 960 × 480 images, more than 100 frames can be processed per second, so that the requirement of the system on real-time performance is met. The surgical instruments are quickly identified and positioned, the model of the brain and the surgical instrument model in the early stage are led into a virtual scene, and the visual navigation of the operation can be realized through image registration.

3. Surgical control system

The operation control system 03 controls the whole process of the ventricular puncture operation, and the operation control system 03 can comprise a computer, for example, wherein the computer comprises an image processing module, a motor control module, a force sense signal processing module, an interactive device communication module and a visual navigation display module. The human-computer cooperation algorithm control flow includes preoperative planning, visual navigation and human-computer cooperation, and a safety guarantee mechanism combining vision and force sense, as shown in fig. 9.

The preoperative planning process includes: obtaining image information of a surgical object; image segmentation; determining a surgical target; a surgical object is registered. In addition to this, there is a need to prepare the surgical environment, including: securing a visual marker (marker) to the surgical object; ensuring that the surgical object is within the field of view of the binocular camera and is fixed; make the operation robot in the working position (visible to the binocular camera)

Firstly, the robot leads the puncture actuator to reach the starting position of puncture under the visual guidance according to the preoperative planned track. The method comprises the steps of detecting the real-time angular points of images of a binocular camera, matching a rapid template based on a visual marker, calculating the position of a puncture operation end effector fixedly connected with the visual marker in real time, and displaying an operation state in real time in host computer software by combining the pose of a mechanical arm.

And after the robot reaches the puncture starting pose, an operator operates the interaction equipment to carry out man-machine cooperative operation. According to the design of the hardware of the puncture actuator, the initial data F of the pressure sensor is obtained after the puncture initial point is reached₁₀，F₂₀. Real-time reading of the pressure sensor data and calculation of the penetration force according to equation (2-3) is then commenced. The penetration force is proportionally fed back to the operator's hand through the interaction device. The operation of an operator in the puncture process has double guarantees of operation navigation visual display and interactive equipment force feedback.

The operator first controls the drill feed via the interaction device until the target position is reached, at which time the drill is not rotated. The operator then pulls the interaction device to trigger the bit to rotate. The operator controls the feed rate reasonably based on the force feedback from the hand. When the drill bit breaks through the skull, the rotation and the feeding of the drill bit are immediately and automatically stopped according to the sudden change of the puncture force. The operator may then perform other operations.

The operator can adjust the operation according to the operation navigation and the force feedback in the whole process of the puncture operation of the human in the ring, the whole process of the puncture is controlled by the operator through the interactive device, and the requirement on the operator is very low. The system can assist a doctor to efficiently complete the puncture operation while fully considering the safety.