阅读说明：本技术 自主移动飞行器检查系统 (Autonomous mobile aircraft inspection system ) 是由 阿梅尔·利亚卡特 洛伊德·汀格勒 菲利普·耶茨 于 2020-06-23 设计创作，主要内容包括：公开了一种自主检查系统,该自主检查系统包括至少一个自主移动机器人。该机器人具有多个二维LiDAR扫描仪,每个扫描仪具有二维扫描平面。多个扫描仪以非共面的扫描平面取向被安装在自主移动机器人上。处理器包括用于接收来自多个LiDAR阵列的点数据的输入以及用于提供检查数据的输出。处理器被配置成：将来自多个LiDAR扫描仪的点数据编译成自主移动机器人的周围的三维图,以及识别三维图内的物品的尺寸和轮廓。还公开了一种检查方法。(An autonomous inspection system is disclosed that includes at least one autonomous mobile robot. The robot has a plurality of two-dimensional LiDAR scanners, each having a two-dimensional scan plane. A plurality of scanners are mounted on the autonomous mobile robot in a non-coplanar scan plane orientation. The processor includes an input for receiving point data from multiple LiDAR arrays and an output for providing inspection data. The processor is configured to: compiling point data from multiple LiDAR scanners into a three-dimensional map of the surroundings of the autonomous mobile robot, and identifying dimensions and contours of items within the three-dimensional map. An inspection method is also disclosed.)

1. An autonomous inspection system comprising:

at least one autonomous mobile robot, wherein the autonomous mobile robot,

the robot has a plurality of two-dimensional LiDAR scanners, each scanner having a two-dimensional scan plane,

the plurality of scanners are mounted on the autonomous mobile robot in a non-coplanar scan plane orientation; and

a processor, the processor comprising:

an input to receive point data from the plurality of two-dimensional LiDAR arrays;

an output for providing inspection data;

the processor is configured to:

compiling point data from the plurality of LiDAR scanners into a three-dimensional map of the surroundings of the autonomous mobile robot, an

Identifying dimensions and contours of the items within the three-dimensional map.

2. The autonomous inspection system of claim 1, wherein each of the plurality of two-dimensional LiDAR scanners detects point data representing a cross-section of a respective plane at which the scanner is aimed.

3. The autonomous inspection system of claim 1 or 2, wherein the plurality of two-dimensional LiDAR scanners have a fixed orientation relative to the autonomous mobile robot.

4. The autonomous inspection system of any preceding claim, wherein the plurality of two-dimensional LiDAR scanners comprises at least one scanner having a scan plane that extends vertically above the robot.

5. The autonomous inspection system of claim 3, wherein a two-dimensional LiDAR scanner pair is arranged with vertically extending scan planes, each scan plane being tilted on opposite sides of an axis perpendicular to the plane of the robot.

6. The autonomous inspection system of claim 5, wherein the two-dimensional LiDAR scanner pairs each rotate relative to a longitudinal axis of the robot.

7. The autonomous inspection system of any preceding claim, wherein the plurality of two-dimensional LiDAR scanners comprises at least one scanner arranged with generally horizontally extending scan planes.

8. The autonomous inspection system of claim 7 comprising a plurality of generally horizontal scanners, each scanner having a scan arc, the arcs aligned to provide 360 degrees of scan coverage around the robot.

9. The autonomous inspection system of any preceding claim, wherein the processor is further configured to combine the point data with stored map data.

10. The autonomous inspection system of any preceding claim, wherein the processor is a processor of the autonomous mobile robot.

11. A method of scanning a fuselage, the method comprising:

providing an autonomous mobile robot having a plurality of two-dimensional LiDAR scanners, the plurality of arrays mounted on the autonomous mobile robot with scan planes in a non-coplanar orientation;

navigating the autonomous mobile robot around a space around the fuselage while scanning the space around the autonomous mobile robot using the LiDAR scanner;

compiling an image of the fuselage according to the point data of the LiDAR scanner.

12. The method of scanning a body of claim 11, further comprising:

providing a map of a space navigated by the autonomous mobile robot; and

compiling the image of the fuselage includes combining the map with point data of the two-dimensional LiDAR scanner.

13. The method of scanning a body of claim 11 or 12, further comprising:

features within an image of the fuselage are identified and the dimensions and/or contours of the scanned features are compared to stored data to mark defects or disqualified portions in the scanned fuselage.

14. A machine-readable storage medium comprising instructions executable by a processor to:

receiving two-dimensional point data from a plurality of LiDAR scans;

stitching the two-dimensional point data to form a three-dimensional map;

combining LiDAR scan data with stored map build data; and

features of the objects within the three-dimensional map are identified, and the dimensions and/or contours of the identified features are compared to mark defects or disqualified portions in the scanned objects.

15. An autonomous mobile robot, comprising:

a plurality of two-dimensional LiDAR scanners each having a fixed two-dimensional scanning plane extending outward from the robot, wherein the plurality of two-dimensional LiDAR scanners are positioned such that a three-dimensional scanning envelope is defined that extends around the autonomous mobile robot, outward in a horizontal direction and a vertical direction, beyond an outer perimeter of the autonomous mobile robot; and

a processing portion configured to receive two-dimensional point data from the plurality of LiDAR scans and stitch the two-dimensional point data to form a three-dimensional map of a space around the autonomous mobile robot.

Technical Field

The invention relates to an autonomous inspection system and a method of autonomous inspection. In particular, but not exclusively, embodiments relate to aircraft structure inspection.

Background

Automation has become increasingly important in manufacturing, for example, to increase manufacturing efficiency or to improve safety. Automated systems are common in many industries, such as automotive manufacturing. However, in some industries, automated systems may be expensive and difficult to implement, for example, aerospace manufacturing (and particularly commercial aircraft manufacturing) where facilities often occupy a large footprint due to the size and scale of the product and the complexity and number of the various manufacturing and assembly processes. For such industries, autonomous systems offer much greater benefits compared to automated systems. An autonomous system may be contrasted with an automated system in that the autonomous system is able to perform tasks with a greater degree of independence, e.g., the autonomous system may be adaptive, able to learn, and have the ability to make "decisions. Thus, an autonomous system may, for example, be able to operate in locations where the system has not been previously used, or in dynamic environments where other systems or items (items) are not in known locations. In contrast, automated systems (e.g., manufacturing robots in automotive manufacturing) will typically be dedicated to performing repetitive tasks in a highly controlled environment.

One area in which the use of autonomous vehicles or autonomous robots may be beneficial is the inspection of aeronautical structures (aerostructures). Such inspections may be required routinely, for example, during aircraft manufacturing (e.g., in the final assembly line) and also at aircraft maintenance, repair and overhaul facilities (which will be referred to herein by the standard acronym "MRO"). Known solutions for automated solutions typically require the use of expensive three-dimensional scanners or cameras, such as 3D light direction and range (LiDAR) scanners. Such a scanner may be used, for example, in a point-of-care positioning and mapping (SLAM) device. To reduce complexity and/or cost, some systems, such as handheld systems, are known to replace 3D scanners with moving 2D scanners (e.g., scanners on flexible mounts that can scan across scanned items). However, such systems may not provide good response times and result in longer inspection times. Furthermore, in an industrial environment, health and safety issues may arise in the use of such systems.

One example of potential automation in Aircraft MRO applications is the "Air-Robot" project, which is developing wheeled Collaborative mobile robots for assisting humans in Inspection activities (and as described, for example, in the paper "Air-Robot: Air Enhanced Inspection by Smart and collectivity Robot", which can be obtained at:http://laris.univ-angers.fr/_resources/IFAC2017/IFAC_Paper_ 3176.pdf). The Air-robot is capable of autonomously performing navigation tasks and routinely performing inspection tasks. The Air-Cobot robot is provided with an array of sensing systems including a navigation sensor including four cameras, two laser rangefinders, a Global Positioning System (GPS) receiver, and an Initial Measurement Unit (IMU), and a non-destructive test sensor including a pan-tilt-zoom (PTZ) camera and a 3D scanner.

At least some embodiments of the present invention aim to provide alternative inspection robots, for example, it may be desirable to provide more cost effective robotic systems or robotic systems with reduced complexity to enable autonomous inspection to be used in a wider range of applications, or to provide systems with better range, accuracy and speed.

Disclosure of Invention

One aspect of the present invention provides an autonomous inspection system, including: at least one autonomous mobile robot having a plurality of two-dimensional LiDAR scanners, each scanner having a two-dimensional scan plane, the plurality of scanners mounted on the autonomous mobile robot in a non-coplanar scan plane orientation; and a processor, the processor comprising: an input to receive point data from a plurality of LiDAR arrays; an output for providing inspection data; the processor is configured to: compiling point data from multiple LiDAR scanners into a three-dimensional map of the surroundings of the autonomous mobile robot, and identifying dimensions and contours of items within the three-dimensional map. In particular, embodiments provide an autonomous mobile aircraft inspection system.

Another aspect of the invention provides a method of scanning a body, the method comprising: providing an autonomous mobile robot having a plurality of two-dimensional LiDAR scanners, the plurality of arrays mounted on the autonomous mobile robot with scan planes in a non-coplanar orientation; navigating the autonomous mobile robot around a space around the fuselage while scanning the space around the autonomous mobile robot using a LiDAR scanner; the image of the fuselage is compiled from the point data of the LiDAR scanner.

Yet another aspect of the invention provides a machine-readable storage medium comprising instructions executable by a processor to: receiving two-dimensional point data from a plurality of LiDAR scans; stitching the two-dimensional point data to form a three-dimensional map; combining LiDAR scan data with stored map build data; and identifying features of the object within the three-dimensional map and comparing the dimensions and/or contours of the identified features to mark defects or disqualified portions (non-conformities) in the scanned object.

Yet another aspect of the present invention provides an autonomous mobile robot, including: a plurality of two-dimensional LiDAR scanners each having a fixed two-dimensional scanning plane extending outward from the robot, wherein the plurality of two-dimensional LiDAR scanners are positioned such that a three-dimensional scanning envelope is defined that extends around the autonomous mobile robot, outward in a horizontal direction and a vertical direction, beyond a perimeter of the autonomous mobile robot; and a process configured to receive two-dimensional point data from a plurality of LiDAR scanners and stitch the two-dimensional point data to form a three-dimensional map of a space around the autonomous mobile robot.

The plurality of scanners mounted on the autonomous mobile robot in a non-coplanar scan plane orientation also have a non-parallel scan plane orientation.

Embodiments of the present invention advantageously enable three-dimensional scanning inspection activities to be performed without the need to provide a dedicated 3D scanner. This provides an advantage in reducing the cost of the scanning system because two-dimensional LiDAR scanners are readily commercially available and at a cost that is significantly lower than commercially available three-dimensional scanning systems. It will be appreciated that the reduction in cost and complexity of the scanning device in embodiments of the invention may therefore enable autonomous scanning to be more broadly implemented.

In particular, the present invention runs counter to conventional approaches in the field using dedicated 3D scanners, as the inventors have surprisingly found that current two-dimensional LiDAR scanners provide sufficiently accurate scanning details (e.g., accuracy as low as 20 μm to 30 μm) to enable reliable three-dimensional maps to be created. Embodiments may provide generated three-dimensional data that is both sufficiently accurate for general inspection, and may even provide data quality that is at least as good as conventional inspection or measurement techniques, such as dedicated high-volume metrology equipment.

Scanning using a two-dimensional LiDAR scanner fixed on an autonomous mobile robotic platform may also enable significant time savings compared to existing or manual or automated measurement systems. For example, it may be noted that a two-dimensional LiDAR scanner may provide accurate position data over an area up to 20m to 25m from the scanner. Thus, it should be appreciated that embodiments may quickly scan large items while also providing high quality data. This is particularly useful in aircraft manufacturing or MRO facilities due to the scale of the aircraft and fuselage structure. Accordingly, embodiments of the present invention may be particularly configured for use in aerospace inspection. For example, for MRO facilities and the like, embodiments of the present invention may be used for surface defect identification and inspection. In aircraft manufacturing applications, embodiments may be used, for example, to qualify the shape of an aircraft structure.

Multiple two-dimensional LiDAR scanners may each detect point data that represents a cross-section of a respective plane at which the scanner is aimed.

A plurality of two-dimensional LiDAR scanners have a fixed orientation relative to the autonomous mobile robot. While this fixed orientation may be adjustable (e.g., during initial configuration), for simplicity, embodiments do not require any movement of the LiDAR scanner relative to the autonomous mobile robot during use. This may help to reduce scan time. For example, a stationary scanner may reduce scan time compared to a system that uses a moving (e.g., translating, tilting, or rotating) two-dimensional scanner to create three-dimensional information.

In an embodiment of the invention, the plurality of two-dimensional LiDAR scanners includes at least one scanner having a scan plane that extends vertically above the robot. This ensures that the system scans an envelope that extends not only horizontally around the autonomous mobile robot but also above the robot and helps to build a complete three-dimensional map. This may be in contrast to methods in which a two-dimensional LiDAR scanner is used only for navigation and/or collision avoidance (such that a scan extending vertically above the robot may be deemed unnecessary).

In particular, embodiments of the present invention include at least one scanner having a scan plane near a vertical axis (or more specifically, an axis perpendicular to a reference plane extending through the autonomous mobile robot). For example, at least one scan plane may be aligned at less than 45 degrees relative to a vertical plane (in other words, the scan plane may have a vertical component that is greater than a horizontal component).

Two-dimensional LiDAR scanner pairs may be arranged with non-coplanar scan planes that extend vertically. For example, each scan plane may be tilted on opposite sides of an axis perpendicular to the plane of the robot. A two-dimensional LiDAR scanner pair may have planes that are at acute angles relative to each other about an axis that is perpendicular to the robot plane. A two-dimensional LiDAR scanner pair having non-coplanar scan planes that extend vertically may also each rotate relative to the longitudinal axis of the robot. Rotation of the scan plane may increase the scan coverage of the envelope around the robot.

The plurality of two-dimensional LiDAR scanners may also include at least one scanner arranged with a generally horizontally extending scan plane (or more specifically, an axis parallel to a reference plane extending through the autonomous mobile robot). In some embodiments, multiple generally horizontal scanners may be provided. For example, each scanner may have a scanning arc (the scanning area of each scanner is typically a sector lying in a scanning plane and extending around the scanner about the arc length defined by the configuration of the scanner). Multiple generally horizontal scanners may be aligned to provide 360 degrees of scanning coverage around the robot.

The processor may also be configured to combine the point data with stored map data. The known location of the autonomous mobile robot relative to the stored map data may be associated with point data for combining the map data with the point data. For example, the known location data may be provided from a navigation system of the autonomous mobile robot. The time stamp data may also be recorded in the dot data and the generated three-dimensional map. For example, temporal data may be used to enable real-time deformation and/or movement of items to be measured (which may be particularly useful in a manufacturing context).

The processor may be configured by including a machine-readable medium including instructions executable by the processor to: compiling point data from multiple LiDAR scanners into a three-dimensional map of the surroundings of the autonomous mobile robot, and identifying dimensions and contours of items within the three-dimensional map (e.g., by comparing the map to stored project data).

The processor may be a processor of an autonomous mobile robot. Alternatively or additionally, the processor may be a centralized processor in communication with the autonomous mobile robot. The centralized processor may be part of a cloud computing system, for example. The central processor may, for example, accumulate data from a plurality of autonomous mobile robots. For example, in some implementations, an inspection system may include multiple autonomous mobile robots according to an implementation, and a processor may compile point data from multiple LiDAR scanners on each mobile robot into a combined three-dimensional map.

Whilst aspects of the invention have been described above, the invention extends to any inventive combination of the features set out above or in the following description.

Drawings

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

FIG. 1 shows a schematic diagram of an autonomous mobile robot for use in an embodiment of the invention;

FIG. 2 shows a schematic diagram of an example navigation map for the autonomous mobile robot of FIG. 1;

FIG. 3 illustrates an example of an autonomous mobile robot in proximity to an object and corresponding scan data generated;

fig. 4 shows a flow chart representing a method according to an embodiment.

Detailed Description

For clarity, the terms "horizontal" and "vertical" are used herein and may be understood as a reference to the general direction relative to the autonomous mobile robot in use and are not intended to be construed narrowly. In particular, it should be understood that the horizontal direction is generally parallel to an x-y reference plane extending through the autonomous mobile robot and substantially parallel to a substrate on which the robot operates (which may not be truly horizontal if the substrate is tilted). Similarly, the vertical direction will be generally parallel to a z-axis extending perpendicular to the x-y reference plane. For the purpose of accurate data capture, the scanning system may have a navigation system for identifying the orientation of the autonomous mobile robot so that data captured relative to the axis of the autonomous mobile robot in use may be corrected to the universal axis.

An autonomous mobile aircraft inspection system 1 according to an embodiment of the invention is shown in fig. 1. The system includes at least one autonomous mobile robot 10. The autonomous mobile robot includes a plurality (four in the illustrated example) of two-dimensional LiDAR (light directing and ranging) scanners 50. Each scanner may be a commercially available safety-rated LiDAR device, each having a two-dimensional scan plane 22, 24, 32, 34. The scanners 50 collectively define a scan envelope that extends in three dimensions around the robot 10. In particular, it may be noted that the scanner is arranged such that the envelope completely surrounds the robot 10 on the horizontal axis and also extends completely around the robot 10 (at least extending upwards from the ground) on the vertical axis. In the example shown, the robot 10 is provided with: a first scanner pair having a scan plane 22 and a scan plane 24 in a generally horizontal plane; and a second scanner pair having a scan plane 32 and a scan plane 34 in a generally vertical plane. Using commercially available two-dimensional security level LiDAR scanners, it has been found that the scan planes 22, 24, 32, 34 can have a range of up to about 20 to 25m, and can provide scan accuracy of about 20 to 30 μm.

The two generally horizontal scan planes 22, 24 are spaced apart in the vertical direction. Scan plane 22 and scan plane 24 may be parallel planes and may be parallel to an x-y reference plane of robot 10. Typically, each scanner has a scan arc that covers a sector of less than 360 degrees in the scan plane. Thus, to minimize or remove any blind spots from the scan, the two scan planes 22 and 24 are rotated relative to each other about the vertical (z) axis. For example, the small sectors of each scan plane 32, 34 that are not scanned may be diametrically opposed.

The two generally vertical scan planes 32 and 34 are disposed at an angle relative to each other to maximize the scan area of the envelope. The scan planes 32 and 34 can each be tilted relative to the vertical axis and to each other, e.g., an acute angle β can be provided between the two scan planes 32 and 34 about the y-axis, such that each scan plane has a tilt of several degrees (e.g., less than 10 degrees) relative to the vertical. The scan plane 32 and the scan plane 34 may also be relatively rotated with respect to the z-axis such that they are angled with respect to the longitudinal axis of the robot 10.

The autonomous mobile robot 10 will include a processor for controlling its navigational movements and scanning. Typically, the robot 10 will also be communicatively coupled to a network (e.g., via a wireless network), and a centralized controller may be provided in the network. Thus, it will be appreciated that depending on the particular system configuration, data from the scanner 50 may preferably be processed by the processor of the robot 10, or whether the data is transmitted in a basic format and processed separately.

As shown in fig. 2, embodiments of the present invention may be positioned with a horizontal scanner pair in a manner similar to existing systems, and fig. 2 shows a typical navigation map in the x-y plane. A map of the previously recorded environment may be provided that includes previously observed items 210, as shown in black pixels in fig. 2. The map may also define an area or boundary 250 to which the robot 10 will not travel. The dark pixels 220 indicate obstacles identified by the navigation-assisted horizontal LiDAR scanner. According to an embodiment of the invention, the robot also acquires additional information by having a scan envelope of the identified object, which is shown in fig. 2 in light grey shading 230.

According to an embodiment, the point data from each scanner may be stitched together and, optionally, combined with stored map data. This enables a detailed 3D map of the environment to be accumulated in a fast and accurate manner.

An example of a multi-scanner approach is shown in fig. 3. This is shown in fig. 3. In FIG. 3A, a photograph shows an autonomous mobile robot 10 with an array of two-dimensional LiDAR scanners 50, which is positioned in front of a hanging obstacle (in the form of a table), according to an embodiment of the present invention. A two-dimensional LiDAR scanner may record a cross-section of the table as shown in FIG. 3B, which may represent, for example, a warning zone 310 and an emergency stop zone 320, as well as a blind spot 340.

By stitching the cross-sections from each scan plane 22, 24, 32, 34 and using the known locations of the autonomous mobile robots within the scan space, embodiments of the present invention can form a three-dimensional point cloud of the environment. Existing two-dimensional maps can be used to form the composite map. The generated map may be used to improve navigation, for example by enabling an automated mobile robot to avoid overhanging objects when planning a route. However, the applicant also confirmed that: the map is accurate enough to also be used to measure objects within the scanning area. Thus, the autonomous mobile robot of an embodiment may navigate around items of an airborne structure and form a three-dimensional map for measuring features of the structure or for inspection and identification of defects or non-conforming portions of shape. The system may, for example, have a machine-readable storage system containing data regarding the airborne structure to be inspected. The three-dimensional map may be compared to stored data to, for example, identify and classify objects and/or defects in objects, and/or the qualification of the dimensions and/or contours of objects to design specifications.

Embodiments of the present invention may provide significant advantages in enabling larger parts, such as aircraft components, to be inspected accurately in a short time and without additional labor costs. This may for example enable more frequent checks than are currently possible. It will also be appreciated that because the system and method of embodiments are capable of tracking multiple points simultaneously, the tracking may enable real-time deformations or movements in the components to be seen during the assembly phase. The system and method of the present invention may save up to 80% of time, for example, compared to conventional methods such as manual measurements or the use of laser trackers or radar.

Embodiments of the invention may further comprise: a machine-readable storage medium comprising instructions executable by a processor. Such instructions may be used to implement the methods of embodiments of the present invention using a suitable autonomous mobile robot.

Although the invention has been described above with reference to a preferred embodiment, it will be appreciated that various modifications may be made without departing from the scope of the invention as defined in the appended claims. For example, while the described embodiments relate to a single autonomous controlled robot, it should be understood that a plurality of such robots may be used in a single system, with network control being used to combine and map the captured data.

Although the embodiments described herein use the term "autonomous mobile robot," it will be understood that this may include other forms of autonomous guided vehicles. For example, in some embodiments of the invention, a multipurpose autonomous vehicle may be utilized to perform scanning using a two-dimensional array of scanners, while also navigating the facility for another purpose (e.g., transporting or moving parts or structures).

Note that the term "or" as used herein is to be interpreted to mean "and/or" unless explicitly stated otherwise.