阅读说明：本技术 机器人化体表病灶区域定位跟踪系统 (Robotized body surface focus area positioning and tracking system ) 是由 丛杨 唐旭 于 2019-01-10 设计创作，主要内容包括：本发明涉及机器人化体表病灶区域定位跟踪系统,该系统可利用人工标识物或皮肤表面特征定位跟踪皮肤病灶。通过检测人工标识物的位姿实现对病灶区域的跟踪；通过特征匹配的方式寻找特征点,利用特征点标记病灶区域并实现对病灶区域的跟踪。本发明能够根据医生的诊断结果,利用在皮肤表层特征自动定位病变区域,利用皮肤表面高精度三维重建技术,实现对病灶区域高精度定位,并结合机器人技术自动完成治疗。通过识别标识的位姿实现对检测的定位跟踪治疗,提升治疗的精准度。(The invention relates to a robot body surface focus area positioning and tracking system, which can position and track skin focuses by using artificial markers or skin surface features. Tracking the focus area by detecting the pose of the artificial marker; and searching for characteristic points in a characteristic matching mode, marking the focus area by using the characteristic points and realizing the tracking of the focus area. According to the invention, the lesion area can be automatically positioned by utilizing the characteristics on the surface layer of the skin according to the diagnosis result of a doctor, the high-precision positioning of the lesion area is realized by utilizing the high-precision three-dimensional reconstruction technology of the surface of the skin, and the treatment is automatically completed by combining the robot technology. The positioning tracking treatment of detection is realized through the pose of the identification mark, and the treatment accuracy is improved.)

1. A robotized body surface lesion area positioning and tracking system is characterized in that the system is used for tracking skin lesions by using artificial markers.

2. The robotic body surface lesion area locating and tracking system of claim 1, comprising:

the image acquisition device is used for acquiring an image containing the marker, identifying the marker and acquiring the pose of the marker; identifying a supposed focus area, and acquiring the position of a focus operating point;

the positioning module is used for obtaining the terminal pose of the robot according to the position and the posture of the plane where the marker is located;

the human-computer interaction system is used for manually screening the lesion area on the surface of the skin marked by the marker;

and the robot control system is used for controlling the robot to move to the operation point so that the robot operates the operation point by obtaining the terminal pose.

3. The robotic body surface lesion area locating and tracking system of claim 2, wherein the acquiring of image information including a marker and the identification of the marker comprises the steps of:

firstly, performing image space conversion on the acquired RGB image information, converting the RGB image information into HSV color space, and selecting an H channel as processing data of the next step;

then, performing threshold processing on the H channel image by adopting a self-adaptive threshold method, and converting the H channel image into a binary image; extracting a binary image contour, and screening the artificial marker by utilizing the geometric characteristic of the artificial marker;

and then, acquiring the position and the posture of the plane of the marker by using the geometric projection relation, and feeding back to the robot control system.

4. The system for positioning and tracking the lesion area on the robotic body surface according to claim 2, wherein the step of identifying the assumed lesion area and obtaining the position of the lesion operating point comprises the steps of:

firstly, collecting a multispectral image of the skin surface marked by an artificial marker, setting a threshold value according to the physical characteristics of skin tissues on the surface layer of a human body under multispectral imaging, carrying out threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the outline, color and focus distribution of the assumed focus area, wherein the obtained skin surface parameters comprise at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

5. A robotized body surface lesion area positioning and tracking system is characterized by being used for positioning and tracking skin lesions by using skin surface features.

6. The robotic body surface lesion area locating and tracking system of claim 5, comprising:

the image acquisition device is used for performing multispectral imaging on the skin surface of the appointed focus detection area to acquire an assumed focus area;

the human-computer interaction system is used for manually setting a focus detection area and manually screening a hypothesis focus area;

the positioning module is used for positioning and tracking a focus treatment point according to the skin surface characteristics around the focus;

and the robot control system is used for controlling the robot to move to an operation point so that the robot operates according to the position and the posture of the focus.

7. The system according to claim 6, wherein the step of performing multispectral imaging of the skin surface on the specified lesion detection area to obtain the assumed lesion area comprises the steps of:

firstly, acquiring a multispectral image of the skin surface in a focus detection area, setting a threshold value according to the physical characteristics of human body surface skin tissues under multispectral imaging, performing threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the contour, color and focus distribution of the focus area to obtain skin surface parameters including at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

8. The system of claim 6, wherein the system for locating and tracking the lesion area on the body surface comprises: and displaying the position of the assumed lesion region obtained by segmentation, and adding the assumed lesion region screened out by a human-computer interaction system into a waiting queue as a lesion region to be treated.

9. The system according to claim 6, wherein the robotic body surface lesion area positioning and tracking system is configured to position the pose of the lesion by using the skin surface features, specifically to cyclically wait for the queue, to complete the operations on all areas in the queue: circulating to the current focus area to be treated, extracting characteristic points at the periphery of the focus, and matching to the corresponding focus area on the current skin surface by using a characteristic matching method; and then, carrying out surface three-dimensional reconstruction on the focus area to be treated, and selecting an operation point in the focus area to be treated, wherein the selection rule is that an acute angle formed by the normal direction of the operation point and the normal direction of a plane obtained by fitting all the characteristic points is the minimum.

10. The system for positioning and tracking the lesion area on the robot body surface according to claim 6, wherein the control robot moves to an operation point for operation, specifically: obtaining the pose of the tail end of the mechanical arm according to the pose of the current focus operation point, and controlling the tail end of the robot to move to the pose; the robot operates the operation point at the pose; the terminal pose of the mechanical arm is that after the terminal of the robot moves to the pose, the optical axis of a lens of an image acquisition device carried by the terminal coincides with the normal of a focus operation point, and a treatment point is positioned in the center of an image.

Technical Field

The invention belongs to the field of medical appliances, and particularly relates to a robotized body surface focus region positioning and tracking system.

Background

With the development of basic medicine and science and technology, many advanced scientific technologies have been transformed into advanced instruments and equipment for treating skin diseases, such as vitiligo with 308nm excimer laser, skin diseases with 311 narrow-spectrum UVB, acne removal with laser, and the like. However, most of the existing devices have several problems:

1. inaccurate positioning: most of the existing treatment equipment places the output end of the instrument at the pathological change position in a manual positioning mode, so that the treatment range is inaccurate, and the treatment precision is insufficient.

2. The lesion area needs to be manually switched: when a plurality of pathological changes need to be treated, the position of the instrument needs to be changed manually mostly, and the efficiency is low.

3. Patient movement results in the lesion area not receiving normal treatment: in the treatment process, the patient inevitably does not move, so that the lesion area moves, and the treatment effect is reduced.

Disclosure of Invention

The invention aims to provide a positioning and tracking system for a robot body surface focus region.

The technical scheme adopted by the invention for realizing the purpose is as follows: a robot body surface focus area positioning and tracking system is used for positioning and tracking a skin focus by utilizing an artificial marker or skin surface characteristics.

Robotized body surface lesion area positioning and tracking system, comprising:

the image acquisition device is used for acquiring an image containing the marker, identifying the marker and acquiring the pose of the marker; identifying a supposed focus area, and acquiring the position of a focus operating point;

the positioning module is used for obtaining the terminal pose of the robot according to the position and the posture of the plane where the marker is located;

the human-computer interaction system is used for manually screening the lesion area on the surface of the skin marked by the marker;

and the robot control system is used for controlling the robot to move to the operation point so that the robot operates the operation point by obtaining the terminal pose.

The collecting of the image information containing the identifier and the identification of the identifier comprises the following steps:

firstly, performing image space conversion on the acquired RGB image information, converting the RGB image information into HSV color space, and selecting an H channel as processing data of the next step;

then, performing threshold processing on the H channel image by adopting a self-adaptive threshold method, and converting the H channel image into a binary image; extracting a binary image contour, and screening the artificial marker by utilizing the geometric characteristic of the artificial marker;

and then, acquiring the position and the posture of the plane of the marker by using the geometric projection relation, and feeding back to the robot control system.

The method for identifying the assumed lesion area and acquiring the position of the lesion operating point comprises the following steps:

firstly, collecting a multispectral image of the skin surface marked by an artificial marker, setting a threshold value according to the physical characteristics of skin tissues on the surface layer of a human body under multispectral imaging, carrying out threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the outline, color and focus distribution of the assumed focus area, wherein the obtained skin surface parameters comprise at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

Robotized body surface lesion area positioning and tracking system, comprising:

the image acquisition device is used for performing multispectral imaging on the skin surface of the appointed focus detection area to acquire an assumed focus area;

the human-computer interaction system is used for manually setting a focus detection area and manually screening a hypothesis focus area;

the positioning module is used for positioning and tracking a focus treatment point according to the skin surface characteristics around the focus;

and the robot control system is used for controlling the robot to move to an operation point so that the robot operates according to the position and the posture of the focus.

The skin surface multispectral imaging is carried out on the appointed focus detection area to obtain an assumed focus area, and the method comprises the following steps:

firstly, acquiring a multispectral image of the skin surface in a focus detection area, setting a threshold value according to the physical characteristics of human body surface skin tissues under multispectral imaging, performing threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the contour, color and focus distribution of the focus area to obtain skin surface parameters including at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

The screening assumes a lesion area, and sets the area to be treated as follows: and displaying the position of the assumed lesion region obtained by segmentation, and adding the assumed lesion region screened out by a human-computer interaction system into a waiting queue as a lesion region to be treated.

The method comprises the following steps of positioning the position and pose of a focus by utilizing skin surface characteristics, specifically, circularly waiting for a queue, and finishing the operation of all areas in the queue: circulating to the current focus area to be treated, extracting characteristic points at the periphery of the focus, and matching to the corresponding focus area on the current skin surface by using a characteristic matching method; and then, carrying out surface three-dimensional reconstruction on the focus area to be treated, and selecting an operation point in the focus area to be treated, wherein the selection rule is that an acute angle formed by the normal direction of the operation point and the normal direction of a plane obtained by fitting all the characteristic points is the minimum.

The control robot moves to an operation point to operate, and specifically comprises the following steps: obtaining the pose of the tail end of the mechanical arm according to the pose of the current focus operation point, and controlling the tail end of the robot to move to the pose; the robot operates the operation point at the pose; the terminal pose of the mechanical arm is that after the terminal of the robot moves to the pose, the optical axis of a lens of an image acquisition device carried by the terminal coincides with the normal of a focus operation point, and a treatment point is positioned in the center of an image.

The invention has the following beneficial effects and advantages:

1. according to the invention, the lesion area can be automatically positioned by utilizing the characteristics on the surface layer of the skin according to the diagnosis result of a doctor, and the high-precision positioning of the lesion area is realized by utilizing the high-precision three-dimensional reconstruction technology on the surface of the skin, and the treatment is automatically completed.

2. According to the invention, the positioning tracking treatment of the detection can be realized by identifying the pose of the mark according to the diagnosis result of the doctor and the detection area mark set by the doctor, and the treatment accuracy is improved.

3. The invention can utilize the high-precision positioning technology to realize the analysis of the skin surface conditions before and after treatment in the same detection area and assist doctors in evaluating the treatment effect.

Drawings

FIG. 1 is a schematic diagram of a robotic body surface lesion area locating and tracking system according to the present invention;

FIG. 2 is a schematic diagram of the positional relationship between the laser therapeutic device and the image data acquisition device;

FIG. 3 is a flow chart of artificial marker assisted lesion location tracking;

FIG. 4 is a schematic view of an artificial marker to aid in lesion location tracking;

FIG. 5 is a focal localization tracking procedure without artificial markers;

FIG. 6 is a schematic view of lesion location tracking without artificial markers.

Detailed Description

The present invention will be described in further detail with reference to examples.

The present embodiment provides a robotic body surface lesion area positioning and tracking system, as shown in fig. 1. Wherein, the hardware includes: the system comprises a camera, a light source, a mechanical arm and laser treatment equipment carried by the mechanical arm; the software comprises a human-computer interaction system for acquiring, processing and analyzing image data and imaging.

The system has the main functions of positioning the focus on the surface of the skin and realizing the automatic auxiliary treatment of the robot. During the treatment process, an operator can monitor the treatment process in real time through the human-computer interaction system and can operate the treatment process, such as stopping, adjusting parameters, recovering and the like. Meanwhile, the human-computer interaction system is responsible for overall data acquisition, processing and analysis, robot operation feedback and control and the like, and displays all information needing to be visualized to an operator in a visual and complete form, so that the operator can comprehensively know the scene condition. In addition, the human-computer interaction system also provides a data storage function, can store data such as treatment parameters, treatment effects, focus data and the like in a grouping manner, and is used for recording the treatment condition of the patient in the whole process. The operator can extract the historical treatment condition of the patient and compare the historical treatment condition with the current condition, and the system can give a numerical reference of the treatment effect according to the comparison result. Therefore, the human-computer interaction system needs manual intervention operation, such as identification region setting, focus region screening, robot operation control and the like.

The image acquisition device is used for acquiring image information of the skin surface, wherein the image information comprises multispectral imaging, surface three-dimensional reconstruction and the like. The multispectral imaging of the skin surface is to obtain the brightness information of different depths of the skin surface by utilizing the physical characteristics of reflection, scattering, absorption and the like of the skin tissue of the human body surface under the multispectral imaging, and the information is stored, processed and analyzed in the form of an image; the skin surface three-dimensional reconstruction is used for carrying out local three-dimensional reconstruction on a skin surface lesion region so as to obtain the position and the posture of the surface of a treatment point.

Referring to fig. 2, fig. 2 is a schematic diagram showing a positional relationship between the laser therapeutic device and the image data acquisition device of the system. The image acquisition device and the laser therapeutic device are fixedly connected together through the curing device, the axes of the image acquisition device and the laser therapeutic device are intersected at the point A, and the position of the point A meets the distance requirement of man-machine safety. During the treatment process, the point A is overlapped with the treatment point, then the laser therapeutic device is extended to the point A through the telescopic mechanism, and the laser treatment process is started; meanwhile, in the process of the movement of the laser therapeutic device, the force applied to the laser therapeutic device is detected in real time to ensure the safety of the human-computer.

The system mainly provides two focus positioning and tracking modes, which are as follows:

the first mode is as follows: the artificial marker is location tracked, see fig. 3. Fig. 3 shows a lesion localization tracking process in this mode, which specifically includes the following steps:

1.1 Artificial marker settings: after the treatment area is diagnosed and selected by the physician, a manual marker is deployed on the lesion surface, which provides one possible approach as shown in fig. 4.

1.2, identifying the marker, and acquiring the pose of the marker: after the marker is set, leading the patient to an image acquisition device, acquiring image information containing the marker and identifying the marker, wherein the identification process comprises the following steps: firstly, performing image space conversion on the acquired RGB image information, converting the RGB image information into HSV color space, and selecting an H channel as processing data of the next step; then, performing threshold processing on the H channel image by adopting a self-adaptive threshold method, and converting the H channel image into a binary image; and extracting the binary image contour, and screening the artificial marker by utilizing the geometric characteristics of the artificial marker. After the marker is identified, the position and the posture of the plane where the marker is located are obtained by utilizing the geometric projection relation and fed back to the robot control system.

And 1.3, controlling the robot to reach the position of the marker by the robot control system. And the robot control system calculates the terminal pose by using the coordinate transformation rule and system calibration, and controls the terminal to reach the designated pose. The appointed pose means that after the tail end of the robot moves to the pose, the image acquisition system carried by the tail end can be aligned to the plane where the artificial mark is located, and the mark center is located in the image center.

1.4 identifying the focus area and obtaining the accurate position of the focus. Firstly, acquiring multispectral information of the skin surface inside the artificial marker by using an image acquisition module, setting a threshold value according to physical characteristics of human body surface skin tissues such as reflection, scattering, absorption and the like under multispectral imaging, performing threshold value segmentation on the multispectral image, extracting a segmentation contour to obtain an assumed focus area, and analyzing the skin surface condition according to the contour, color and focus distribution of the focus area, such as focus area proportion, focus level and the like; then, a local surface three-dimensional reconstruction technology is utilized to carry out surface three-dimensional reconstruction on the focus area, and a point with the smallest acute angle between the normal direction of the inner surface of the focus area and the normal of the artificial marker is selected as a treatment point.

1.5 the operator confirms and completes the treatment process. After the position of the focus is determined, the treatment can be started only after an operator confirms the focus position in the human-computer interaction system without error, and in the treatment process, the operator can control the system in real time through the human-computer interaction system so as to ensure the accuracy and the safety of the treatment process.

And a second mode: the operator selects the area by the man-machine interactive system, and the focus area is positioned by the surrounding image characteristics. As shown in fig. 5, the specific process is as follows:

2.1 multispectral imaging and lesion area segmentation. An operator sets a focus detection area by a human-computer interaction system, and then performs multispectral imaging on the skin surface of the appointed focus detection area to acquire a supposed focus area, which can be specifically referred to as 1.4.

2.2 the operator screens the area of the hypothetical lesion and sets the area to be treated. The position of the assumed lesion area obtained by segmentation is displayed in the human-computer interaction system, relevant parameters of the obtained lesion are analyzed, an operator screens the assumed lesion area according to images, analysis data and the like, and the area to be treated is added in a treatment waiting queue. The treatment waiting queue is a focus area image queue which can be set by an operator and can be treated by a current mode, and only focus areas and skin surface images in a specified neighborhood of the focus areas are in the images.

2.3 using the skin surface characteristics to position the focal position. And circulating a treatment waiting queue to finish the treatment of the whole area. The specific treatment process is as follows: circulating to the current focus area with treatment, extracting feature points such as SIFT, SURF and the like at the periphery of the focus, and matching to the corresponding focus area on the current skin surface by using a feature matching method; and then, carrying out surface three-dimensional reconstruction on the focus region by utilizing a local surface three-dimensional reconstruction technology, and selecting a treatment point in the focus region, wherein the selection rule is that the acute angle between the normal direction of the treatment point and the normal direction of a plane obtained by fitting all the characteristic points is minimum.

2.4 the robot moves to the treatment point and keeps a man-machine safe distance. And the robot control system calculates the pose of the tail end of the mechanical arm according to the pose of the current focus treatment point and controls the tail end to move to the pose. The pose of the tail end of the mechanical arm means that after the tail end of the robot moves to the pose, the axis of an image acquisition system carried by the tail end is superposed with the normal of a focus treatment point, and the treatment point is located in the center of an image, as shown in fig. 6.

2.5 the operator confirms and completes the treatment process. See 1.5.

In the mode I, the focus area is tracked by detecting the pose of the artificial marker; and for the second mode, searching a characteristic point in a characteristic matching mode, marking a focus area by using the characteristic point and realizing the tracking of the focus area.