阅读说明：本技术 基于视觉度量的建筑外墙表面喷涂方法 (Building exterior wall surface spraying method based on visual measurement ) 是由 贺王鹏 陈志杰 廖楠楠 支云彭 李�诚 郭宝龙 于 2020-06-08 设计创作，主要内容包括：本发明涉及一种基于视觉度量的建筑外墙表面喷涂方法,具体含有以下步骤：1、选择一个标定好的CCD相机和一个测距仪；2、将相机和测距仪连接在建筑外墙喷涂机器人控制电路上；3、拍摄建筑外墙表面图像并测量相机到建筑外墙表面的距离；4、建筑外墙喷涂机器人根据相机到建筑外墙表面的距离值、相机焦距和建筑外墙表面图像的像面尺寸,计算出建筑外墙表面图像的实际尺寸；5、训练U-Net网络；6、将建筑外墙表面图像输入到U-Net网络,进行图像分割,划分出需进行喷涂的建筑外墙表面区域；7、对需喷涂的外墙表面区域进行喷涂；本发明处理速度快,计算量小,目标窗口轮廓测量精度高,很好地满足了实际施工需求。(The invention relates to a building outer wall surface spraying method based on visual measurement, which specifically comprises the following steps: 1. selecting a calibrated CCD camera and a distance meter; 2. connecting a camera and a range finder on a control circuit of a spraying robot for the outer wall of a building; 3. shooting an image of the surface of the outer wall of the building and measuring the distance from a camera to the surface of the outer wall of the building; 4. the building outer wall spraying robot calculates the actual size of the building outer wall surface image according to the distance value from the camera to the building outer wall surface, the camera focal length and the image surface size of the building outer wall surface image; 5. training a U-Net network; 6. inputting the surface image of the building outer wall into a U-Net network, carrying out image segmentation, and dividing a surface area of the building outer wall needing spraying; 7. spraying the surface area of the outer wall to be sprayed; the method has the advantages of high processing speed, small calculated amount and high measurement precision of the target window profile, and well meets the actual construction requirements.)

1. A building exterior wall surface spraying method based on visual measurement is characterized by comprising the following steps: comprises the following steps:

step one, selecting a CCD camera and a range finder, and calibrating the CCD camera;

secondly, an image signal input interface and a distance signal input interface are arranged on a control circuit of the building outer wall spraying robot, a signal output interface of the CCD camera is connected with the image signal input interface, and a signal output interface of the range finder is connected with the distance signal input interface;

thirdly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the building outer wall to obtain an image of the surface of the building outer wall, and the control circuit of the building outer wall spraying robot controls the distance meter to measure the distance from the CCD camera to the surface of the building outer wall to obtain a distance value from the CCD camera to the surface of the building outer wall;

step four, calculating the actual size of the building outer wall surface image by the control circuit of the building outer wall spraying robot according to the distance value from the CCD camera to the surface of the building outer wall, the focal length of the CCD camera and the image surface size of the building outer wall surface image;

step five, training the U-Net network: the control circuit of the building outer wall spraying robot inputs the Wireframe data set and the York data set into a U-Net network for network learning;

step six, the control circuit of the building outer wall spraying robot inputs the building outer wall surface image obtained in the step three into a trained U-Net network, image segmentation is carried out, and the building outer wall surface area needing spraying is divided;

and step seven, controlling the building outer wall spraying robot to spray the surface area of the building outer wall needing to be sprayed by the building outer wall spraying robot according to the actual size of the image of the surface of the building outer wall calculated in the step four by the control circuit of the building outer wall spraying robot.

2. The building exterior wall surface spraying method based on the visual measurement as claimed in claim 1, wherein: and seventhly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the sprayed building outer wall to obtain an image of the surface of the sprayed building outer wall, then the image of the surface of the sprayed building outer wall is subjected to image conversion in a Lab color space by using a superpixel segmentation algorithm, and then the spraying quality is evaluated.

3. The building exterior wall surface spraying method based on the visual measurement as set forth in claim 1 or 2, wherein: the CCD camera is a planar array CCD camera, and the range finder is a phase laser range finder.

4. The building exterior wall surface spraying method based on the visual measurement as set forth in claim 1 or 2, wherein: the image signal input interface and the distance signal input interface are USB interfaces, a control circuit of the building outer wall spraying robot comprises a central processing unit, a signal conversion module, a USB interface module and a power supply module, the image signal input interface and the distance signal input interface are connected with an input port of the signal conversion module through the USB interface module, an output port of the signal conversion module is connected with the central processing unit, and the power supply module supplies power to the central processing unit, the signal conversion module and the USB interface module.

5. The building exterior wall surface spraying method based on the visual measurement as claimed in claim 4, wherein: the USB interface module adopts a GL850G chip.

The technical field is as follows:

the invention relates to a building outer wall surface spraying method, in particular to a building outer wall surface spraying method based on visual measurement.

(II) background art:

the vigorous development of social economy, particularly the rapid development of the building industry, more and more building outer wall facades need to be sprayed and painted, the workload of construction workers is large, the construction workers are in high altitude, and the personal safety of the workers cannot be well guaranteed. With the continuous improvement of building technology, various intelligent high-altitude operation robots are continuously applied to the building industry, so that the work of people is effectively replaced, the cost of the building industry is greatly reduced, and workers are liberated from hard, heavy and high-risk work. The application of the robot for spraying the exterior wall of the building can realize standardization and process of the construction of the facade of the exterior wall of the building and improve the spraying quality and the operation efficiency of the facade of the exterior wall of the building. Generally, the outer wall facade of a building is a main element needing spraying, and the outer wall facade of the building has more special surfaces such as windows and balconies, so how to quickly and accurately judge whether the area needs spraying construction, automatically and accurately measure the shape and the accurate size of the area and whether the spraying quality reaches the expectation becomes a bottleneck limiting the large-area continuous construction of the spraying robot on the outer wall facade of the building.

The existing spraying robot measuring system is used for acquiring a building image, identifying the image and detecting color difference based on machine vision, and determining that the image size and the window size in the image can be decomposed into the detection of a straight line segment in a two-dimensional image.

However, there are many problems in actual construction, which prevent the complete automation of the spraying process:

(1) limitation of contour extraction. Only the target contour of a specific shape can be extracted, the target with a complex boundary cannot be processed, and the extracted contour has larger false detection and missing detection compared with the actual target edge.

(2) Target profile sizing limitations. The target profile measurement accuracy is higher for a particular shape and lower for other particular target profiles.

(3) Is sensitive to background interference. When scene light changes or more noise points and miscellaneous spots exist on the wall surface, the boundary detection precision and the identification of the foreground area are easily influenced.

(4) The algorithm is less efficient. The pixel-based detection algorithm needs to count the global information, which greatly increases the calculation amount and causes difficulty in meeting the actual construction requirements.

(III) the invention content:

the technical problem to be solved by the invention is as follows: the building outer wall surface spraying method based on the visual measurement is high in processing speed, small in calculated amount and high in target window contour measurement precision, and well meets the actual construction requirements.

The technical scheme of the invention is as follows:

a building exterior wall surface spraying method based on visual measurement comprises the following steps:

selecting a CCD camera and a range finder according to the running environment of the system, wherein the CCD camera requires good imaging quality and high flexibility, and carrying out camera calibration on the CCD camera to ensure minimum image distortion and prepare for subsequent size measurement;

step two, arranging an image signal input interface and a distance signal input interface on a control circuit of the building outer wall spraying robot, connecting a signal output interface of a CCD camera with the image signal input interface, connecting a signal output interface of a range finder with the distance signal input interface, integrating two hardware devices, and ensuring the transmission of image and distance data;

thirdly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the building outer wall to obtain an image of the surface of the building outer wall, and the control circuit of the building outer wall spraying robot controls the distance meter to measure the distance from the CCD camera to the surface of the building outer wall to obtain a distance value from the CCD camera to the surface of the building outer wall;

step four, in the non-contact measurement of the camera, a corresponding triangulation method is provided according to the camera imaging principle, and the actual size of the building outer wall surface image is calculated by the building outer wall spraying robot control circuit according to the distance value from the CCD camera to the building outer wall surface, the CCD camera focal length and the image surface size of the building outer wall surface image by utilizing the triangle similarity principle or according to the lens field angle;

step five, training the U-Net network: the building outer wall spraying robot control circuit inputs the Wireframe data set and the York data set into the U-Net network for network learning so as to improve the detection performance and the detection speed of the U-Net network on images; the U-Net network has a very obvious effect in image segmentation as a U-shaped symmetric depth convolution neural network, the left depth and the right depth of the U-Net network structure are generally 4 layers, compression processing is carried out for 4 times by utilizing convolution and pooling, and then the size of a feature diagram is expanded by carrying out deconvolution and deconvolution operation; the straight line segment detection by using the network has high accuracy but low speed, the information of the background image of the outer wall is simple, the U-Net network can still well operate under a small sample, and the speed problem of the original network is solved by reducing the encoding and decoding times in the network; the experimental result proves that the algorithm after learning and improvement has higher detection performance on the obtained simple image, and the measurement error can meet the requirement on the surface measurement of the building outer wall;

step six, the control circuit of the building outer wall spraying robot inputs the building outer wall surface image obtained in the step three into a trained U-Net network, image segmentation is carried out, and the building outer wall surface area needing spraying is divided;

and step seven, controlling the building outer wall spraying robot to spray the surface area of the building outer wall needing to be sprayed by the building outer wall spraying robot according to the actual size of the image of the surface of the building outer wall calculated in the step four by the control circuit of the building outer wall spraying robot.

And seventhly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the sprayed building outer wall to obtain an image of the surface of the sprayed building outer wall, then the image of the surface of the sprayed building outer wall is subjected to image conversion in a Lab color space by using a superpixel segmentation algorithm, and then the spraying quality is evaluated.

The CCD camera is a planar array CCD camera, and the range finder is a phase laser range finder.

The image signal input interface and the distance signal input interface are both USB interfaces, the control circuit of the building outer wall spraying robot comprises a central processing unit, a signal conversion module, a USB interface module and a power supply module, the image signal input interface and the distance signal input interface are connected with the input port of the signal conversion module through the USB interface module, the output port of the signal conversion module is connected with the central processing unit, and the power supply module supplies power to the central processing unit, the signal conversion module and the USB interface module; the design of the voltage stabilizing circuit of the power supply module ensures the power supply requirements of the CCD camera and the range finder, and the signal conversion module fuses multimode signals of the camera and the range finder.

The USB interface module adopts GL850G chip.

The invention has the beneficial effects that:

1. the invention obtains the color information of the area to be sprayed by the CCD camera, obtains the depth information of the area to be sprayed by the laser range finder, performs fusion processing, combines with the full convolution neural network U-Net to segment the building outer wall image obtained by the CCD camera, determines the window edge and the window size in the image and the surface area of the building outer wall needing to be sprayed, realizes non-contact measurement, has high system processing speed, small calculated amount and high target window contour measurement precision, and well meets the actual construction requirement.

2. The invention adopts a CCD camera with small distortion in a planar array mode, calibrates the camera before measurement, eliminates the influence of the distortion of the camera on the image, and can be used for acquiring the high-quality image; the depth information is obtained by adopting a phase laser range finder, the measuring distance is short, and the precision reaches millimeter level; the trained full convolution neural network U-Net has strong detection performance and high speed on the straight line segment, and the measurement precision and the detection speed of the system on the outline size of the target window are greatly improved.

3. The LSD line detection method provided by the invention takes the line segment detection problem as the regional coloring extraction problem, performs image semantic segmentation to extract lines through the U-Net network, improves the U-Net network aiming at the characteristics of single color and less internal texture of the surface image of the building outer wall, and provides a shallow U-Net network model on the basis of the original model for extracting the low-dimensional information of the image, so that the convolution operation for extracting the depth information is reduced, the network detection rate is also improved, the jump connection times in the network are increased for increasing the weight ratio of the low-dimensional information, and the trained network can better extract the shallow information in the outer wall image.

4. The invention adopts a superpixel segmentation algorithm to evaluate the spraying quality, and utilizes the Lab color space to carry out image conversion, so that the spraying quality and effect are closer to the color perception of human in the Lab color space compared with RGB.

(IV) specific embodiment:

the building exterior wall surface spraying method based on the visual measurement comprises the following steps:

selecting a planar array CCD camera and a phase laser range finder according to the running environment of the system, selecting an industrial lens with the focal length of 6mm by using a CCD camera lens, wherein the CCD camera has good imaging quality and high flexibility, performing camera calibration on the CCD camera, ensuring the minimum image distortion and preparing for subsequent dimension measurement;

step two, arranging an image signal input interface and a distance signal input interface on a control circuit of the building outer wall spraying robot, connecting a signal output interface of a CCD camera with the image signal input interface, connecting a signal output interface of a range finder with the distance signal input interface, integrating two hardware devices, and ensuring the transmission of image and distance data;

thirdly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the building outer wall to obtain an image of the surface of the building outer wall, and the control circuit of the building outer wall spraying robot controls the distance meter to measure the distance from the CCD camera to the surface of the building outer wall to obtain a distance value from the CCD camera to the surface of the building outer wall;

step four, in the non-contact measurement of the camera, a corresponding triangulation method is provided according to the camera imaging principle, and the actual size of the building outer wall surface image is calculated by the building outer wall spraying robot control circuit according to the distance value from the CCD camera to the building outer wall surface, the CCD camera focal length and the image surface size of the building outer wall surface image by utilizing the triangle similarity principle or according to the lens field angle;

step five, training the U-Net network: the building outer wall spraying robot control circuit inputs the Wireframe data set and the York data set into the U-Net network for network learning so as to improve the detection performance and the detection speed of the U-Net network on images; the U-Net network has a very obvious effect in image segmentation as a U-shaped symmetric depth convolution neural network, the left depth and the right depth of the U-Net network structure are generally 4 layers, compression processing is carried out for 4 times by utilizing convolution and pooling, and then the size of a feature diagram is expanded by carrying out deconvolution and deconvolution operation; the straight line segment detection by using the network has high accuracy but low speed, the information of the background image of the outer wall is simple, the U-Net network can still well operate under a small sample, and the speed problem of the original network is solved by reducing the encoding and decoding times in the network; the experimental result proves that the algorithm after learning and improvement has higher detection performance on the obtained simple image, and the measurement error can meet the requirement on the surface measurement of the building outer wall;

step six, the control circuit of the building outer wall spraying robot inputs the building outer wall surface image obtained in the step three into a trained U-Net network, image segmentation is carried out, and the building outer wall surface area needing spraying is divided;

and step seven, controlling the building outer wall spraying robot to spray the surface area of the building outer wall needing to be sprayed by the building outer wall spraying robot according to the actual size of the image of the surface of the building outer wall calculated in the step four by the control circuit of the building outer wall spraying robot.

And seventhly, the control circuit of the building outer wall spraying robot controls the CCD camera to shoot the surface of the sprayed building outer wall to obtain an image of the surface of the sprayed building outer wall, then the image of the surface of the sprayed building outer wall is subjected to image conversion in a Lab color space by using a superpixel segmentation algorithm, and then the spraying quality is evaluated.

The image signal input interface and the distance signal input interface are both USB interfaces, the control circuit of the building outer wall spraying robot comprises a central processing unit, a signal conversion module, a USB interface module and a power supply module, the image signal input interface and the distance signal input interface are connected with the input port of the signal conversion module through the USB interface module, the output port of the signal conversion module is connected with the central processing unit, and the power supply module supplies power to the central processing unit, the signal conversion module and the USB interface module; the design of the voltage stabilizing circuit of the power supply module ensures the power supply requirements of the CCD camera and the range finder, and the signal conversion module fuses multimode signals of the camera and the range finder. The USB interface module adopts GL850G chip.