阅读说明：本技术 智能塔吊物料运输路径设置方法和装置 (Intelligent tower crane material transportation path setting method and device ) 是由 陈德木 蒋云 陈曦 陆建江 赵晓东 顾姣燕 于 2021-07-20 设计创作，主要内容包括：本申请实施例提供一种基于三维空间模型的智能塔吊物料运输路径设置方法和装置。该方法包括：根据至少两个摄像头的视频监控数据进行双目视觉交叉建模,建立塔吊周边环境的三维空间模型；根据挂钩和待运输物料的三维坐标,计算并设置将待运输物料运送至挂钩下的第一路径任务；计算挂钩下移距离,并控制挂钩下降下移距离以挂取待运输物料；计算并设置将待运输物料运送至终点三维坐标的第二路径任务,并将第二路径任务发送给塔吊,以控制塔吊按照第二路径任务将待运输物料运送至终点三维坐标处。本申请能够根据运输任务全自动、无人化操作的将物料运输到挂钩下面,自动控制挂钩将物料挂起,并控制塔吊将物料运输到终点位置,实现了塔吊物料运输的无人化、智能化。(The embodiment of the application provides a method and a device for setting a material transportation path of an intelligent tower crane based on a three-dimensional space model. The method comprises the following steps: performing binocular vision cross modeling according to the video monitoring data of at least two cameras, and establishing a three-dimensional space model of the tower crane surrounding environment; calculating and setting a first path task for conveying the material to be transported to the position below the hook according to the three-dimensional coordinates of the hook and the material to be transported; calculating the downward moving distance of the hook, and controlling the downward moving distance of the hook to hang and take the materials to be transported; and calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the terminal, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the terminal according to the second path task. This application can be according to the full-automatic, unmanned operation of transportation task with the material transport below the couple, automatic control couple hangs the material to the control tower crane transports the material to the terminal position, realized tower crane material transport unmanned, intelligent.)

1. The utility model provides an intelligent tower crane material transportation path setting method based on three-dimensional space model, which is characterized by comprising the following steps:

installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras, and establishing a three-dimensional space model of the surrounding environment of the tower crane;

marking the positions of a tower crane, a hook and a material to be transported in the three-dimensional space model, and obtaining three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation;

receiving and analyzing transportation task instruction information of the material to be transported, and calculating and marking a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model;

calculating and setting a first path task for conveying the material to be conveyed to the position below the hook according to the three-dimensional coordinates of the hook and the material to be conveyed, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to convey the material to be conveyed to the position right below the hook according to the first path task;

calculating the downward moving distance of the hook, and controlling the hook to descend by the downward moving distance to hang and take the material to be transported;

and calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task.

2. The method of claim 1,

carrying out binocular vision cross modeling according to the video monitoring data of the at least two cameras, and establishing a three-dimensional space model of the tower crane surrounding environment, wherein the method comprises the following steps:

constructing a disparity map by utilizing the real-time image of the tower crane;

carrying out graying processing and wavelet denoising processing on the disparity map in sequence to obtain a processed disparity map;

determining the spatial layout of the tower crane in a single direction according to the disparity map in the single direction;

and splicing the disparity maps in multiple directions based on an image splicing algorithm to construct a three-dimensional space model image of the tower crane.

3. The method of claim 2,

the position of marking tower crane, couple and waiting to transport the material in the three-dimensional space model to obtain through pixel decomposition calculation the three-dimensional coordinate of tower crane, couple and waiting to transport the material includes:

marking the positions of a tower crane, a hook and a material to be transported in the three-dimensional space model;

calculating the number of pixels of the tower crane, the hook and the material to be transported relative to the origin of coordinates of the three-dimensional space model in the images shot by the two cameras through pixel decomposition;

calculating according to the proportional relation between the number of pixels and the size of a preset tower crane to obtain two-dimensional coordinates of the tower crane, the hook and the material to be transported in two camera images;

and performing three-dimensional space coupling according to the two-dimensional coordinates to obtain three-dimensional coordinates of the tower crane, the hook and the material to be transported.

4. The method of claim 3,

the method for receiving and analyzing the transportation task instruction information of the material to be transported, calculating and marking the end point three-dimensional coordinate of the material to be transported in the three-dimensional space model comprises the following steps:

receiving and analyzing transportation task instruction information of a material to be transported to obtain transportation destination information;

marking the position of a transportation terminal in the three-dimensional space model;

calculating the number of pixels of the transportation terminal relative to the origin of coordinates of the three-dimensional space model in images shot by the two cameras through pixel decomposition;

calculating according to the proportional relation between the number of pixels and the size of a preset tower crane to obtain two-dimensional coordinates of the transportation terminal point in two camera images;

and performing three-dimensional space coupling according to the two-dimensional coordinates to obtain a three-dimensional coordinate of the transportation terminal, wherein the three-dimensional coordinate is used as a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model.

5. The method of claim 4,

according to the three-dimensional coordinates of the hook and the material to be transported, calculating and setting a first path task for transporting the material to be transported to the position below the hook, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to transport the material to be transported to the position right below the hook according to the first path task, the method comprises the following steps:

calculating the three-dimensional coordinate of the material to be transported to a first transport position right below the hook according to the three-dimensional coordinates of the hook and the material to be transported, and taking a straight path between the initial position of the material to be transported and the first transport position as a first path task;

sending the first path task to an unmanned transport vehicle, and analyzing the first path task by the transport vehicle to obtain an initial position and a first transport position of the material to be transported;

and controlling the transport vehicle to transport the material to be transported to the first transportation position from the initial position according to the first path task.

6. The method of claim 5,

the calculation couple moves down the distance to control the couple and descend move down the distance in order to hang get treat the transportation material, include:

calculating the number of pixels of the distance between the hook and the material in an image shot by any one of the two cameras through pixel decomposition;

calculating the distance between the hook and the material to be transported according to the proportional relation between the number of pixels and the size of a preset tower crane, and taking the distance as the downward movement distance of the hook;

and controlling the hook to descend the downward moving distance and hanging the material to be transported.

7. The method of claim 6,

the calculating and setting of the second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and the sending of the second path task to the tower crane are performed to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task, and the method comprises the following steps:

taking an arc-shaped path between the three-dimensional coordinate of the first transportation position and the three-dimensional coordinate of the destination as a second path task;

sending the second path task to a tower crane, and analyzing the second path task by the tower crane to obtain a three-dimensional coordinate of a first transportation position of the material to be transported and a three-dimensional coordinate of a terminal point;

and controlling the tower crane to convey the material to be conveyed to the destination three-dimensional coordinate from the first conveying position according to the second path task.

8. The utility model provides an intelligence tower crane material transport route sets up device based on three-dimensional space model which characterized in that includes:

the three-dimensional space modeling module is used for installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras and establishing a three-dimensional space model of the surrounding environment of the tower crane;

the coordinate marking module is used for marking the positions of the tower crane, the hook and the material to be transported in the three-dimensional space model and obtaining the three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation;

the task analysis module is used for receiving and analyzing the transportation task instruction information of the material to be transported, and calculating and marking a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model;

the first path setting module is used for calculating and setting a first path task for conveying the material to be conveyed to the position below the hook according to the three-dimensional coordinates of the hook and the material to be conveyed, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to convey the material to be conveyed to the position right below the hook according to the first path task;

the hanging control module is used for calculating the downward moving distance of the hook and controlling the hook to descend by the downward moving distance so as to hang and take the material to be transported;

and the second path setting module is used for calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor to implement the method according to any of claims 1-7.

Technical Field

The application relates to the technical field of intelligent tower cranes, in particular to a method and a device for setting a material transportation path of an intelligent tower crane based on a three-dimensional space model.

Background

At present, the tower crane is basically operated and controlled by personnel in a central control room on the tower crane, or is remotely operated and controlled in real time through operators. In the tower crane industry, the current development direction is unmanned tower cranes and intelligent tower cranes, so that a lot of technical problems can be encountered in the industrial upgrading process.

At present, some tower cranes do not need to be directly controlled by people on the tower cranes, can be remotely controlled by equipment such as a control lever and a computer of a remote control room, still belong to the category of controlling the tower cranes by people, and cannot realize full automation of the material transportation task of the tower cranes.

Disclosure of Invention

In view of the above, the application aims to provide a method and a device for setting a material transportation path of an intelligent tower crane based on a three-dimensional space model.

Based on the above purpose, the application provides an intelligent tower crane material transportation path setting method based on a three-dimensional space model, which comprises the following steps:

installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras, and establishing a three-dimensional space model of the surrounding environment of the tower crane;

marking the positions of a tower crane, a hook and a material to be transported in the three-dimensional space model, and obtaining three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation;

receiving and analyzing transportation task instruction information of the material to be transported, and calculating and marking a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model;

calculating and setting a first path task for conveying the material to be conveyed to the position below the hook according to the three-dimensional coordinates of the hook and the material to be conveyed, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to convey the material to be conveyed to the position right below the hook according to the first path task;

calculating the downward moving distance of the hook, and controlling the hook to descend by the downward moving distance to hang and take the material to be transported;

and calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task.

Preferably, the binocular vision cross modeling is performed according to the video monitoring data of the at least two cameras, and a three-dimensional space model of the tower crane surrounding environment is established, including:

constructing a disparity map by utilizing the real-time image of the tower crane;

carrying out graying processing and wavelet denoising processing on the disparity map in sequence to obtain a processed disparity map;

determining the spatial layout of the tower crane in a single direction according to the disparity map in the single direction;

and splicing the disparity maps in multiple directions based on an image splicing algorithm to construct a three-dimensional space model image of the tower crane.

Preferably, the marking of the positions of the tower crane, the hook and the material to be transported in the three-dimensional space model and the obtaining of the three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation comprise:

marking the positions of a tower crane, a hook and a material to be transported in the three-dimensional space model;

calculating the number of pixels of the tower crane, the hook and the material to be transported relative to the origin of coordinates of the three-dimensional space model in the images shot by the two cameras through pixel decomposition;

calculating according to the proportional relation between the number of pixels and the size of a preset tower crane to obtain two-dimensional coordinates of the tower crane, the hook and the material to be transported in two camera images;

and performing three-dimensional space coupling according to the two-dimensional coordinates to obtain three-dimensional coordinates of the tower crane, the hook and the material to be transported.

Preferably, the receiving and analyzing the transportation task instruction information of the material to be transported, and calculating and marking the end point three-dimensional coordinate of the material to be transported in the three-dimensional space model includes:

receiving and analyzing transportation task instruction information of a material to be transported to obtain transportation destination information;

marking the position of a transportation terminal in the three-dimensional space model;

calculating the number of pixels of the transportation terminal relative to the origin of coordinates of the three-dimensional space model in images shot by the two cameras through pixel decomposition;

calculating according to the proportional relation between the number of pixels and the size of a preset tower crane to obtain two-dimensional coordinates of the transportation terminal point in two camera images;

and performing three-dimensional space coupling according to the two-dimensional coordinates to obtain a three-dimensional coordinate of the transportation terminal, wherein the three-dimensional coordinate is used as a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model.

Preferably, the calculating and setting of the first path task for conveying the material to be transported to the position under the hook according to the three-dimensional coordinates of the hook and the material to be transported and the sending of the first path task to an unmanned transport vehicle are performed to control the transport vehicle to convey the material to be transported to the position under the hook according to the first path task, and the calculating and setting of the first path task include:

calculating the three-dimensional coordinate of the material to be transported to a first transport position right below the hook according to the three-dimensional coordinates of the hook and the material to be transported, and taking a straight path between the initial position of the material to be transported and the first transport position as a first path task;

sending the first path task to an unmanned transport vehicle, and analyzing the first path task by the transport vehicle to obtain an initial position and a first transport position of the material to be transported;

and controlling the transport vehicle to transport the material to be transported to the first transportation position from the initial position according to the first path task.

Preferably, the calculating of the downward moving distance of the hook and the controlling of the hook to descend the downward moving distance to hang and take the material to be transported includes:

calculating the number of pixels of the distance between the hook and the material in an image shot by any one of the two cameras through pixel decomposition;

calculating the distance between the hook and the material to be transported according to the proportional relation between the number of pixels and the size of a preset tower crane, and taking the distance as the downward movement distance of the hook;

and controlling the hook to descend the downward moving distance and hanging the material to be transported.

Preferably, the calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task includes:

taking an arc-shaped path between the three-dimensional coordinate of the first transportation position and the three-dimensional coordinate of the destination as a second path task;

sending the second path task to a tower crane, and analyzing the second path task by the tower crane to obtain a three-dimensional coordinate of a first transportation position of the material to be transported and a three-dimensional coordinate of a terminal point;

and controlling the tower crane to convey the material to be conveyed to the destination three-dimensional coordinate from the first conveying position according to the second path task.

Based on above-mentioned purpose, this application has still provided an intelligence tower crane material transport route setting device based on three-dimensional space model, includes:

the three-dimensional space modeling module is used for installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras and establishing a three-dimensional space model of the surrounding environment of the tower crane;

the coordinate marking module is used for marking the positions of the tower crane, the hook and the material to be transported in the three-dimensional space model and obtaining the three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation;

the task analysis module is used for receiving and analyzing the transportation task instruction information of the material to be transported, and calculating and marking a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model;

the first path setting module is used for calculating and setting a first path task for conveying the material to be conveyed to the position below the hook according to the three-dimensional coordinates of the hook and the material to be conveyed, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to convey the material to be conveyed to the position right below the hook according to the first path task;

the hanging control module is used for calculating the downward moving distance of the hook and controlling the hook to descend by the downward moving distance so as to hang and take the material to be transported;

and the second path setting module is used for calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task.

In general, the advantages of the present application and the experience brought to the user are:

this application can be accurate the material carry position of control unmanned intelligent tower crane, according to the full-automatic, the unmanned operation of transportation task with the material transport below the couple, automatic control couple hangs the material to control the tower crane transports the material to the terminal point position, realized tower crane material transport unmanned, intelligent.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.

Fig. 1 shows a schematic diagram of the device architecture of the present application.

FIG. 2 shows a flow chart of an intelligent tower crane material transportation path setting method based on a three-dimensional space model according to an embodiment of the application.

Fig. 3 shows a structural diagram of an intelligent tower crane material transportation path setting device based on a three-dimensional space model according to an embodiment of the application.

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 5 is a schematic diagram of a storage medium provided in an embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 shows a schematic diagram of the device architecture of the present application. In the embodiment of this application, equipment includes the tower crane, treats transportation material, couple, camera, server, places the unmanned transport vechicle (not shown in the figure, the form is not limited, preferred unmanned intelligent transport vechicle) etc. that treats the transportation material. Performing binocular vision cross modeling according to the video monitoring data of at least two cameras, and establishing a three-dimensional space model of the tower crane surrounding environment; calculating and setting a first path task for conveying the material to be transported to the position below the hook according to the three-dimensional coordinates of the hook and the material to be transported; calculating the downward moving distance of the hook, and controlling the downward moving distance of the hook to hang and take the materials to be transported; and calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the terminal, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the terminal according to the second path task. This application can be according to the full-automatic, unmanned operation of transportation task with the material transport below the couple, automatic control couple hangs the material to the control tower crane transports the material to the terminal position, realized tower crane material transport unmanned, intelligent.

FIG. 2 shows a flow chart of an intelligent tower crane material transportation path setting method based on a three-dimensional space model according to an embodiment of the application. As shown in fig. 2, the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model comprises the following steps:

step 101: installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras, and establishing a three-dimensional space model of the surrounding environment of the tower crane;

in this embodiment, specifically, step 101 includes:

constructing a disparity map by utilizing the two real-time images of the tower crane;

carrying out graying processing and wavelet denoising processing on the disparity map in sequence to obtain a processed disparity map;

determining the spatial layout of the tower crane in a single direction according to the disparity map in the single direction;

and splicing the disparity maps in multiple directions based on an image splicing algorithm to construct a three-dimensional space model image of the tower crane.

In a specific embodiment, the imaging of a single camera in the binocular vision system is described by a pinhole camera mathematical model, that is, the projection position Q of any point Q in the image is the intersection point of the connecting line of the optical center and the point Q and the image plane, and the point Q in the physical world has coordinates (X, Y, Z). The projection is a point (x, y, f).

Lens distortion vectors can be simultaneously solved in the camera calibration process, and lens distortion is corrected. The stereo calibration is a process of calculating the geometric relationship between two cameras in space, namely, a rotation matrix R and a translation matrix T between the two cameras are searched, an image black-and-white chessboard image is calibrated, the chessboard image is translated and rotated in front of the cameras in the calibration process, the corner point positions on the chessboard image are obtained at different angles, the rotation matrix R and the translation matrix T between the stereo images are given, and the stereo calibration is carried out by using a related algorithm, for example, a Bouguet algorithm, and the purpose of the stereo calibration is to enable corresponding matching points of images shot by two visual sensors to be respectively in the same-name pixel rows of the two images, so that the matching search azimuth is limited in one pixel row.

The origin of coordinates of the ideal binocular vision stereo coordinate system is the projection center of the left camera, the X axis points to the projection center of the right camera from the origin, the Z axis is perpendicular to the imaging plane of the camera and points to the front, and the Y axis is perpendicular to the arrow of the X-Z plane and points downwards.

Stereo matching is required for the corrected camera to generate a disparity map, for example, stereo matching is performed by selecting a regional gray scale correlation method.

After the graying processing, the parallax image is subjected to wavelet filtering processing, so that the noise of the parallax image is reduced; constructing a three-dimensional space model image of the robot through a stitching algorithm for the current direction disparity map, wherein the stitching algorithm is an image stitching algorithm based on Fourier transform; for example, the algorithm for splicing three-dimensional space images of the tower crane in two adjacent directions performs two-dimensional discrete Fourier transform on two digital images to be spliced.

The method has the advantages that the disparity map in the visual field range can be determined by using the binocular vision system, the layout structure in a single direction can be integrated into an integral space structure by adopting a splicing algorithm, a data fusion algorithm and the like according to the single reverse space layout characteristic of the disparity map, and thus the three-dimensional space model of the tower crane is obtained.

Step 102: marking the positions of a tower crane, a hook and a material to be transported in the three-dimensional space model, and obtaining three-dimensional coordinates of the tower crane, the hook and the material to be transported through pixel decomposition calculation;

in this embodiment, for example, the positions of the tower crane, the hook, and the material to be transported are first marked in the three-dimensional space model. The marking process can be manually marked, and can also be automatically marked in the established three-dimensional space model according to the positions of the captured images of the tower crane, the hook and the material to be transported in the whole model through computer 3D modeling software.

And calculating the number of the tower crane, the hook and the material to be transported relative to the origin of coordinates of the three-dimensional space model in the images shot by the two cameras through pixel decomposition. The purpose of this step is to calculate the number of pixels of each component in the three-dimensional coordinate model relative to the origin through pixel-level analysis to prepare for the next calculation.

And calculating according to the proportional relation between the number of pixels and the size of the preset tower crane to obtain two-dimensional coordinates of the tower crane, the hook and the material to be transported in the two camera images. Because the physical size of the tower crane can be known in advance, the number of pixels obtained by calculation in the previous section is divided by the number of pixels of the tower crane and then multiplied by the physical size of the pixels, and then two-dimensional coordinates of the tower crane, the hook and the material to be transported in two camera images can be obtained.

And performing three-dimensional space coupling according to the two-dimensional coordinates to obtain three-dimensional coordinates of the tower crane, the hook and the material to be transported. In this step, the binocular vision principle is still utilized, for example, three-dimensional coupling calculation is performed on two-dimensional coordinates of the tower crane in the camera A and two-dimensional coordinates of the tower crane in the camera B, and the two-dimensional plane coordinates are converted into a three-dimensional stereo coordinate. For example, through this step, the three-dimensional coordinates of the tower crane are calculated to be (X1, Y1, Z1), the three-dimensional coordinates of the hook are (X2, Y2, Z2), and the three-dimensional coordinates of the material to be transported are (X3, Y3, Z3).

Step 103: and receiving and analyzing the transportation task instruction information of the material to be transported, and calculating and marking the three-dimensional coordinate of the end point of the material to be transported in the three-dimensional space model.

In this embodiment, for example, the server receives, for example, a hoisting task execution requirement of the user from the cloud, and requires to transport the material to be transported into a container that is not far from the tower crane. Firstly, the server analyzes a transportation task instruction to obtain a transportation task, namely, the transportation task is to transport the materials into the container. And then searching a three-dimensional coordinate position corresponding to the container in the three-dimensional space model to be used as a terminal three-dimensional coordinate of the material to be transported in the three-dimensional space model. The process of this search is similar to that in step 102.

For example, the position of the container is first marked in the three-dimensional space model. This marking process can be done manually or automatically by computer 3D modeling software in an already established three-dimensional model of the space, depending on where the captured image of the container is located in the entire model.

And calculating the number of pixels of the container relative to the coordinate origin of the three-dimensional space model in the images shot by the two cameras respectively through pixel decomposition. The purpose of this step is to calculate the number of pixels of each component in the three-dimensional coordinate model relative to the origin through pixel-level analysis to prepare for the next calculation.

And calculating two-dimensional coordinates of the container in the two camera images according to the proportional relation between the number of pixels and the size of a preset tower crane. The physical size of the tower crane can be known in advance, so that two-dimensional coordinates of the container in two camera images can be obtained by dividing the pixel number obtained by calculation in the previous section by the pixel number of the tower crane and multiplying the pixel number by the physical size of the pixel.

And carrying out three-dimensional space coupling according to the two-dimensional coordinates to obtain the three-dimensional coordinates of the container. In this step, the binocular vision principle is still used, for example, the two-dimensional coordinates of the container in the camera a and the two-dimensional coordinates of the container in the camera B are respectively subjected to three-dimensional coupling calculation, and the two-dimensional plane coordinates are converted into a three-dimensional stereo coordinate. For example, by this step, the three-dimensional coordinates of the container are calculated (X4, Y4, Z4).

Step 104: and calculating and setting a first path task for conveying the material to be conveyed to the position below the hook according to the three-dimensional coordinates of the hook and the material to be conveyed, and sending the first path task to an unmanned transport vehicle so as to control the transport vehicle to convey the material to be conveyed to the position right below the hook according to the first path task.

In this embodiment, for example, according to the position coordinates (X2, Y2, Z2) of the hook and the initial position (X3, Y3, Z3) of the material to be transported, three-dimensional coordinates (X2, Y2, Z3) of a first transport position where the material to be transported needs to be transported right below the hook are calculated, and a linear path between the initial position (X3, Y3, Z3) of the material to be transported and the first transport position (X2, Y2, Z3) is used as a first path task;

sending the first path task to an unmanned transport vehicle, and analyzing the first path task by the transport vehicle to obtain an initial position (X3, Y3, Z3) of the material to be transported as a starting point and a first transport position (X2, Y2, Z3) as an end point of the first path task;

and controlling the transport vehicle to transport the material to be transported from the initial position (X3, Y3, Z3) to the first transport position (X2, Y2, Z3) according to the first path task.

Step 105: and calculating the downward moving distance of the hook, and controlling the hook to descend by the downward moving distance to hang and take the material to be transported.

In the embodiment, for the same reason, the number of pixels of the distance between the hook and the material is calculated in the image shot by any one of the two cameras through pixel decomposition;

calculating the distance between the hook and the material to be transported according to the proportional relation between the number of pixels and the size of a preset tower crane, and taking the distance as the downward movement distance of the hook;

and controlling the hook to descend the downward moving distance and hanging the material to be transported.

Step 106: and calculating and setting a second path task for conveying the material to be conveyed to the three-dimensional coordinate of the end point, and sending the second path task to the tower crane so as to control the tower crane to convey the material to be conveyed to the three-dimensional coordinate of the end point according to the second path task.

In the present embodiment, for example, an arc-shaped path between the three-dimensional coordinates (X2, Y2, Z3) of the first transportation position and the end three-dimensional coordinates (X4, Y4, Z4) is taken as the second path task;

sending the second path task to a tower crane, and analyzing the second path task by the tower crane to obtain a three-dimensional coordinate (X2, Y2, Z3) of a first transportation position of the material to be transported and a three-dimensional coordinate (X4, Y4, Z4) of a terminal point;

and controlling the tower crane to convey the material to be conveyed to the destination three-dimensional coordinates (X4, Y4 and Z4) from a first conveying position (X2, Y2 and Z3) according to the second path task.

This application can be accurate the material carry position of control unmanned intelligent tower crane, according to the full-automatic, the unmanned operation of transportation task with the material transport below the couple, automatic control couple hangs the material to control the tower crane transports the material to the terminal point position, realized tower crane material transport unmanned, intelligent.

An application embodiment provides an intelligent tower crane material transportation path setting device based on a three-dimensional space model, and the device is used for executing the method for setting the intelligent tower crane material transportation path based on the three-dimensional space model, as shown in fig. 3, the device comprises:

the three-dimensional space modeling module 501 is used for installing at least two cameras around the unmanned intelligent tower crane, performing binocular vision cross modeling according to video monitoring data of the at least two cameras, and establishing a three-dimensional space model of the surrounding environment of the tower crane;

a coordinate marking module 502, configured to mark positions of a tower crane, a hook, and a material to be transported in the three-dimensional space model, and obtain three-dimensional coordinates of the tower crane, the hook, and the material to be transported through pixel decomposition calculation;

the task analysis module 503 is configured to receive and analyze transportation task instruction information of the material to be transported, and calculate and mark a three-dimensional coordinate of a terminal point of the material to be transported in the three-dimensional space model;

a first path setting module 504, configured to calculate and set a first path task for transporting the material to be transported to a position under the hook according to the three-dimensional coordinates of the hook and the material to be transported, and send the first path task to an unmanned transport vehicle, so as to control the transport vehicle to transport the material to be transported to a position right below the hook according to the first path task;

the hanging control module 505 is used for calculating the downward moving distance of the hook and controlling the hook to descend by the downward moving distance to hang the material to be transported;

and the second path setting module 506 is configured to calculate and set a second path task for conveying the material to be transported to the three-dimensional coordinate of the end point, and send the second path task to the tower crane, so as to control the tower crane to convey the material to be transported to the three-dimensional coordinate of the end point according to the second path task.

The intelligent tower crane material transportation path setting device based on the three-dimensional space model and the intelligent tower crane material transportation path setting method based on the three-dimensional space model provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as methods adopted, operated or realized by application programs stored in the device.

The embodiment of the application also provides electronic equipment corresponding to the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model, so as to execute the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model. The embodiments of the present application are not limited.

Referring to fig. 4, a schematic diagram of an electronic device provided in some embodiments of the present application is shown. As shown in fig. 4, the electronic device 2 includes: the system comprises a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the storage 201 stores a computer program which can be run on the processor 200, and when the processor 200 runs the computer program, the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model provided by any one of the previous embodiments of the present application is executed.

The Memory 201 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the apparatus and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, etc. may be used.

Bus 202 can be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used for storing a program, the processor 200 executes the program after receiving an execution instruction, and the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model, disclosed by any embodiment of the application, can be applied to the processor 200, or implemented by the processor 200.

The processor 200 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 200. The Processor 200 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the method in combination with the hardware thereof.

The electronic equipment provided by the embodiment of the application and the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the electronic equipment.

Referring to fig. 5, the computer readable storage medium is an optical disc 30, and a computer program (i.e., a program product) is stored on the optical disc, and when the computer program is executed by a processor, the method for setting an intelligent tower crane material transportation path based on a three-dimensional space model according to any of the foregoing embodiments is executed.

It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memories (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical and magnetic storage media, which are not described in detail herein.

The computer-readable storage medium provided by the embodiment of the application and the method for setting the material transportation path of the intelligent tower crane based on the three-dimensional space model provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as methods adopted, operated or realized by application programs stored by the computer-readable storage medium.

It should be noted that:

the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the creation apparatus of a virtual machine according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present application, and these should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.