阅读说明：本技术 使用激光线的多摄像头成像系统 (Multi-camera imaging system using laser lines ) 是由 F·皮劳德 P·安托利内斯 O·波雷 于 2020-03-18 设计创作，主要内容包括：本发明涉及一种成像系统,该系统使用传统摄像头和单激光线在转换机或印刷机的输出端执行质量控制。该系统使用分布在印刷片材宽度上的多个摄像头。由于激光线,它可以重构印刷品的完整图像,即使片材不完全平坦或高度不同,也可以补偿几何和光度失真。传统摄像头的使用导致具有成本效益的系统。(The present invention relates to an imaging system that uses a conventional camera and a single laser line to perform quality control at the output of a converting machine or printing press. The system uses multiple cameras distributed across the width of the printed sheet. Due to the laser line, it can reconstruct a complete image of the print, even if the sheets are not perfectly flat or highly different, compensating for geometric and photometric distortions. The use of conventional cameras results in a cost-effective system.)

1. An imaging system for a device configured to translate a flat support at a height above a reference plane in a direction of motion above the plane, the system comprising:

a laser source configured to project a laser line on the support surface, the laser line being in a transverse direction compared to the direction of movement,

-a processor for processing the data received from the receiver,

two cameras with overlapping fields of view arranged along the laser line,

wherein each camera is configured to take an image of a support surface comprising a laser line,

the processor is configured to:

-calculating for each image the height of the support surface along said line using the position of said line in the image,

in each image, a sub-region before or after the laser line is selected depending on the direction of motion,

-combining overlapping or adjacent sub-regions into one common image slice using the calculated heights, an

-outputting the image slice.

2. The imaging system of claim 1, having as many laser sources as cameras, wherein each camera is configured to capture its respective laser line to calculate the height of the support along the line.

3. The imaging system of any preceding claim, wherein the system is configured to synchronize the capturing of images with the translation of the support, thereby combining image slices captured at different times into a single reconstructed image; the system is configured to output the reconstructed image.

4. The imaging system of any of the preceding claims, further comprising an illumination device configured to cause the support flash to flash, wherein a flash duration is configured to be shorter than an exposure time of the camera.

5. A method for merging images from multiple cameras for a device having a flat support that translates at a height above a reference plane in a direction of motion, above the plane; the method comprises the following steps:

-projecting a laser line on the support surface in a lateral direction compared to the direction of movement by using a laser source,

-for each camera

-taking an image of the support surface including the laser line,

-calculating the height of said support along the laser line using the position of the laser line within the image,

-selecting a predetermined area in the image before or after the laser line depending on the direction of movement,

re-interpolating the selected area into a representation corresponding to a completely flat supported image taken at a predetermined height,

-connecting the selected region into the image slice by connecting the regions side by side.

6. The method of claim 5, wherein the re-interpolating step further comprises interpolation based on image coordinates along the support surface and photometric correction based on support height along the laser line.

7. The method of claim 5 or 6, further comprising

-synchronizing the taking of the images with the translation of the support and combining the resulting image slices into a common reconstructed image representative of the support without any gaps or any repetitions.

8. The method of any of claims 5-7, wherein the re-interpolation step uses the height of the support along the laser line taken by the same camera at different times.

9. The method according to any of claims 7-8, wherein the translation of the support is measured by comparing the content of consecutive and overlapping images taken by at least one camera of the system.

10. The method of claim 9, further comprising translating the output of the measurement.

11. The method of any of claims 5-10, further comprising flashing a support with an illumination device while capturing an image of the support, wherein a duration of the flash is shorter than a duration of illumination of the laser line during the exposure of the image.

Technical Field

The present invention is in the field of imaging systems for quality control systems in machines for transporting webs or sheets of material, such as paper or cardboard.

Background

With a printing press or converting machine, a series of single sheets travels through the various units of the machine. In some machines, a web replaces individual sheets. To ensure proper quality, a console may be placed at the end of the printing line. The quality console consists of an imaging system that can generate images of the sheeting. The image is inspected by a control or inspection algorithm to determine if the sheet meets specifications. In other words, it checks for defects that may occur during the printing/converting process.

Today's systems may need to detect small defects on wide sheets. Therefore, multiple cameras are typically required to check the quality of the entire sheet. Prior art imaging systems use fast linear cameras to capture the sheet. The cameras take a single line (or 2-3 lines) of sheets at high speed, and as the sheets travel under the cameras, they are able to take an entire sheet (or an entire web if a web is used).

If the sheet does not travel under the camera at the expected instantaneous speed, it can cause distortion in the image produced. The linear camera itself cannot detect this phenomenon.

In addition, in order to limit the size of the linear camera sensor, the camera uses a projection optical system. Thus, if the sheet is not perfectly flat, i.e. its surface varies in distance from the sensor, it results in a local variation of the image scale.

Today, linear cameras are more expensive than two-dimensional cameras (using matrix-shaped sensors) which become cheaper than linear cameras.

Disclosure of Invention

The present invention relates to a method of using a two-dimensional camera in an imaging system of a quality console to obtain an inexpensive system that solves the above-mentioned problems of the prior art. The invention also relates to an imaging system implementing said method.

The imaging system (and method) is well suited for use with machines configured to transport planar supports, such as printing presses, die cutting machines, windsurfing machines, and more generally converting machines. The support takes the form of a single sheet of material or a web of paper. The top support surface travels in the apparatus at a given height above a reference plane (which may be a real surface or simply a virtual plane when a web is used). In principle, the support travels on a plane, so the height is equal to the support thickness. However, in practice, the height may be variable, for example, when the sheet is flying over a surface, or when the support is not perfectly flat.

The imaging system includes a laser source that projects laser lines on a support surface. There are at least two cameras recording the support, arranged side by side (along the laser line) to potentially photograph the entire width of the support. The laser line must be visible in each image to determine the height of the support along the line. Due to the height information, the images transmitted by the cameras are combined/stitched into image slices, which ideally depict the entire width of the support. The resulting image slices can be used for quality control. Thus, before combining/stitching, sub-regions that do not contain laser lines are selected in each image and used for the combining/stitching operation. Note that to output image slices that accurately depict the entire width of the support, the fields of view of the cameras must partially overlap. Furthermore, for optimum accuracy, a height is required, since it affects the distance of the camera head from the support and thus the reproduction scale of the support in the recorded image.

The imaging system may be built using more cameras than laser sources, or may be built using one laser source per camera, resulting in a module that can be replicated along the width of the support.

By repeating the above-mentioned taking of image slices, we can combine the slices to obtain an image of the entire support (or at least an image of the support whose height is greater than a single image slice). To do this, we can use the support velocity provided by the conversion machine, or use the vertical overlap between image slices to calculate the translation of the support between the two slices (or a combination of both). To calculate support translation using vertical overlap, we ensure that the support carries sufficient texture information, e.g., a set of markers on the border or support. We also oversample the acquisition of the image slices compared to the (maximum) support velocity, ensuring that the image slices do not intersect. Vertical overlap refers to overlap according to the direction of sheet motion. Since the illumination of the support may vary with height, the method may advantageously correct the photometric information of each pixel. The correction is a function of the pixel coordinates and the associated support height.

The sub-region used to construct the image slice does not contain a laser line, so the height measured along the line is not exactly equal to the height of the sub-region. In view of this phenomenon, we can advantageously use the height of the laser line from the previous image shot to calculate the height information in the sub-area. We can also combine the heights of the laser lines from multiple image shots.

Drawings

Embodiments of the invention are illustrated by way of example in the drawings, in which like reference numerals indicate the same or similar elements, and in which;

FIG. 1 shows an example of implementing an imaging system using two cameras and a single laser source;

FIG. 2 shows a sheet, a laser line, the fields of view of two cameras, and an inspection area;

FIG. 3 illustrates the principle of re-interpolation of sheets having different heights towards a sheet at a reference height using orthogonal projections;

FIG. 4 illustrates the principle of re-interpolation of sheets having different heights towards a sheet at a reference height using the unrolling of the sheets;

FIG. 5 shows an example of reconstructing a bicycle image printed on a sheet by a system having two cameras;

fig. 6 shows several examples of exposure timing of the camera, illumination and laser lines.

Detailed Description

Fig. 1 shows an example of an embodiment in which two cameras 3, 4 are used to capture the entire width of a sheet 2 being inspected. The laser source 5 projects a light ray 6 (laser line) onto the sheet 2. The laser line is visible in the image taken by the camera 3 and the image taken by the camera 4.

Note that the projection plane 7 of the laser light has an angle compared to the optical axis 9 of the camera. This may result in the lines being shifted in the coordinate system of the image taken by the camera when the height of the sheet changes. Thus, by measuring the position of the line in the image and knowing the geometric configuration of the camera and laser source, we can calculate the sheet height along the line. In theory, the sheet should be flat. However, in practice it may be curved or wavy. If the height of the top surface of the sheet is changed, the (local) proportion of the imaged surface will also change. Therefore, in order to check the accuracy of the print on the sheet, or the accuracy of the cutting or creasing lines, it is necessary to compensate for this phenomenon.

To compensate for the change in scale due to the change in height, the local height 8 of the sheet is measured along the laser line 6. We assume that the height does not vary along the direction of motion 51 of the sheet. Preferably, to simplify the calculations, the cameras 3, 4 are oriented so that the laser line 6 appears level on the image (for a perfectly flat sheet) and the direction of movement of the sheet 51 generally follows the columns of the image. In this case, we get the height value of each column of the image. More generally, a laser line in a transverse direction compared to the direction of motion of the sheet is sufficient (e.g., with an angle from 45 to 135 degrees); the direction of the image rows/columns is not critical compared to the above.

Fig. 2 shows the sheet seen from above with the fields of view 30, 40 of the cameras 3, 4, respectively. We refer to the direction of motion of the sheet as the direction of motion 51 and the direction of the laser line (for a flat sheet) as the transverse direction. The fields of view of the cameras overlap slightly in the transverse direction (e.g. they may overlap by 5%, 10% or 20%; the percentage of overlap may vary with the height of the sheet). The laser line 6 passes through the fields of view of the two cameras. The deviation of the laser line 6 from a straight line is exaggerated in the figure for illustrative purposes. The camera has an examination area 31, 41 which does not contain a laser line. The inspection area is used to image the sheets and stitched together in the transverse direction to form a single image at the output. The inspection area is located below or above the laser line, i.e. before or after the laser line in the direction of movement of the sheet.

Fig. 3 shows how the image corresponding to a completely flat sheet for a given sheet local height 8 is recalculated. In this example, two manifestations of the sheet are shown: one curved, corresponding to the real case (with exaggerated deformation), and one on the ground, corresponding to the reference case. The choice of the reference height 88 is arbitrary, but may advantageously correspond to a typical situation. Here we assume that the reference sheet is a perfectly flat sheet with the surface 29 at the predetermined height 88. Fig. 3B shows an image of the scene in fig. 3A taken by a camera. The camera performs a projection so that points 12 and 13 in the scene map to the same point 130 in the image (which are projected in the direction 11 in the scene). To correct for scale, we virtually project the sheeting onto the reference height 29 in an orthogonal manner (i.e., along the line of height 8), thereby obtaining an equivalent orthogonal projection in which the scale of the image does not change with distance from the camera. We refer to the resulting image as a re-interpolated image. For example, point 12 is virtually projected on point 14. The point 14 appears at a position 140 on the image. Therefore, to obtain an image of a perfectly flat sheet, we take the pixel value at location 130 and move it to location 140. Note that location 140 is the intersection of line 110 connecting the optical center 100 and location 130 of the image and motion line 104 of projection 14 corresponding to point 12. This process may be repeated for each point of the image; the transformation is determined by the local height, which is constant along the motion lines 102, 104 in the image. Generally, in this example, the function is a mapping from a projection representation to an orthogonal representation, made possible by a measurement of local height. In practice, we use the inverse of this function: for each position H of the re-interpolated image, we compute a transform corresponding to position H to position L and fill the pixel (of the re-interpolated image) at position H with the pixel value (obtained by interpolation, since L may be located between pixels) at position L in the captured image. Therefore, the re-interpolation image is re-interpolation of a captured image corresponding to an image of a completely flat support (sheet) captured at a predetermined height. Calculating a re-interpolated image for each camera; the re-interpolated images are stitched together to form a generic and wider stitched image as output for further processing (e.g. for quality control). An inspection area is defined at a reference height 29 to ensure proper coverage of the entire sheet width (the sheet width being defined perpendicularly from the direction of motion). Note that we neglect any compensation of optical distortion that was corrected due to previous camera calibration steps, which is fairly standard and is beyond the scope of this disclosure.

Since the height variation is small in practice, the line image printed on the support is mapped as a line in the stitched image. Furthermore, due to the orthogonal representation, there is no discontinuity on the lines at the connection points between the re-interpolated images used to construct the stitched image.

As a slightly more accurate but computationally more intensive alternative, the sheet may be virtually unfolded (from the optical center to the edges) rather than orthogonal projection. This method is illustrated in fig. 4. Rather than projecting dots 12 perpendicularly onto dots 14, the sheet is unfolded so that dots 12 are projected onto dots 15. For this purpose, the expansion starts from the optical center 10. It starts with a proxel 17, which proxel 17 corresponds to the next point where a height value is available. The point 17 is projected along a (circular) arc 20 centered on the point 10. Point 17 is projected onto the ground 29, resulting in point 19 (for simplicity of explanation we assume point 10 is here located on the reference height 29). Note the difference between line 22 connecting points 17 and 19 and line 16 following the camera's natural projected light (so points 17 and 18 will appear at the same location in the captured image). Fig. 4C shows this operation or the zoom of fig. 4A. The next point 12 is projected using an arc centered at point 17. Point 12 is projected onto point 24 and point 24 is at the same height as point 17. The projection 14 is obtained by projecting the points 24 along parallel lines 23 onto the line 22. The calculation is repeated for each height value along the laser line and applied to each pixel of the examination region, thereby producing a re-interpolated image. The interpolation process is the same as the example of fig. 3, except for the coordinate calculation.

The image forming apparatus disclosed herein is suitable for use in a printer or a converter. For example, it may be placed at the output of a printing press or at the output of a flat bed press. The image forming apparatus is coupled with a conveying system that translates the sheet under the image forming apparatus and conveys translation information to the image forming apparatus. Depending on the machine, the maximum support (or sheet) width may vary, thus requiring a different number of cameras. For example, a narrow machine may be equipped with two cameras, while a wider machine may be equipped with 3 to 9 cameras in the transverse direction.

Advantageously, rather than using a single laser source for many cameras, one laser source may be selected for each camera. This allows building a combined camera and laser source module that can be replicated to cover the entire sheet width, ensuring quality independent of the maximum support width. This is also logically advantageous from the production and calibration. The method of calculating the height of the sheet remains the same.

Advantageously, in addition to the geometric re-interpolation of the examination region as described above, we can correct the photometric information of each pixel of the examination region. When the height of the support changes, the distance to the lighting device for illuminating the support and the distance to the camera head may change. This can result in a change in the perceived brightness of the camera. In other words, if the height varies, the same support portion will result in different pixel values. To compensate for this phenomenon, we record images of a reference flat surface with constant reflectivity at two different heights: top height and bottom height (e.g., at minimum and maximum expected heights). Then, for each pixel of the two reference images, we calculate one correction factor (for each color of the image) to map the recorded pixel values to a single reference value (corresponding to the constant reflectivity of the reference support). This produces two images that are made up of correction factors (rather than color values). Let us call these images luminance factor images. When recording a new image of the support surface, we calculate the support surface height for each pixel position and calculate the correction factor by interpolating the value from the top luminance factor image and the value from the bottom factor image based on the pixel height compared to the bottom and top height. In practice, this is based on an interpolation of the three-dimensional coordinates applied to each color channel of the image. This correction corrects for illumination variations caused by the height of the support surface, but also corrects for other phenomena such as camera vignetting or illumination non-uniformity.

Note that we may record the reflectivity of the reference flat surface at more than two heights and interpolate between the two to obtain more accurate corrections. However, we found that in view of our illumination settings, two measurements are sufficient.

Fig. 5 shows a shooting process of the imaging system. The left image shows the imaging system taking a sheet with a bike drawn. The right figure shows the same sheet after a moment at the next moment of shooting by the imaging system. This example shows an imaging system with two cameras. The left camera has a field of view 30 and an examination region 31. The right camera has a field of view 40 and an examination region 41. The pictures show the image 32 taken by the left camera and the picture 42 taken by the right camera. These pictures contain an image of the laser line 6. These images are then re-interpolated into pictures 33 and 43 and combined into a stitched picture 34. This process is repeated after a few moments and is displayed on the right side of the picture. The stitched pictures 34, 340 taken at different times are combined/stacked into a reconstructed image 35. The stacking is repeated to obtain an image of the complete sheet 2. Strictly speaking, we do not necessarily need the full height of the sheet: we need only be high enough to perform the subsequent quality control operations. The frequency of the pictures taken is synchronized with the movement 50 of the sheet. If properly processed, the resulting reconstructed image will not show any gaps or any repetitions compared to the image/pattern present on the support (sheeting).

Note that the height of the laser line used to re-interpolate the picture before stitching may be the height from the same picture. However, we prefer to use one of the previous pictures. In practice, we store the height information until the portion of the sheet illuminated by the laser reaches the inspection area. For example, in FIG. 5, we use the height information recorded at the configuration depicted on the left (recorded at time T-1) and apply it to the image on the right of the figure (recorded at time T). In fact, on the left side of the figure, the laser line touches the top of the bicycle transmission shown on the sheet, while on the right side of the figure, the same top of the bicycle transmission is located in the center of the inspection area. Note that we can also take any value between the two lines, or use two lines: one above and one below the examination region and inserting height information between the two lines to obtain a height value for each pixel of the examination region.

In order to obtain a laser image that is visible for any reflectivity of the support surface, one may wish to expose the examination area and the laser line in different ways. The exposure time of the examination area should be kept short. In fact, the movement of the support during exposure should not exceed a fraction of the pixels to avoid motion blur. The exposure time of the examination area refers to the time the camera is "on" (i.e. recording a picture) and the illumination device illuminates the examination area. In practice we keep the camera exposure time longer than the "flash" exposure time, i.e. the time between the moments when the illumination is on and off (here we consider the flash exposure during the image exposure, if the flash is on after the end or exposure, or before its start, we do not calculate it). The exposure time of the examination area is the time when the flash is on and the camera is on. The exposure time of the laser line is not limited by the same constraints (the exposure time of the laser line is the time the laser is on and the camera is on). It does not matter if the support moves during exposure as long as the height of the support does not change during exposure (i.e. the support can move more than one pixel without problems). Therefore, to ensure that the laser line is visible with less power used by the laser (thereby saving equipment cost), the exposure time of the laser line is selected to be longer than the exposure time of the inspection area. Fig. 6 shows an example of exposure timing. The exposure of the camera is indicated by line 60, the exposure of the illumination (i.e. "flash") is indicated by line 61, and the exposure of the laser is indicated by line 62. Position 64 represents a closed or closed position and position 65 represents an open or open position. The exposure time of the laser line may be 5 to 20 times longer, for example 10 times longer, than the flash exposure time. For example, the exposure time of the camera (60) may be set to 800 microseconds, and the flash exposure time (61,66,67) may be set to 80 microseconds. The laser line may be on all the time as in the example of fig. 6, or synchronized (or nearly synchronized) with the exposure 60 of the camera. The on/off exposure curves 66 and 67 of the illumination represent equivalent alternatives to the example on/off exposure curve 61. The abscissa of fig. 6 represents time.

Note that by using picture 32 and comparing it with picture 320 taken at a later time, one can use some image processing algorithm that takes advantage of the overlap between pictures to calculate the motion of the sheet. The calculation may be used to verify that the translation of the sheet 50 meets specifications. Preferably, the sheet may be printed with some indicia on the border to ensure the robustness of the method. Furthermore, in cases where the motion of the support is not uniform or different than expected, the motion calculations can be used to properly combine the image slices.

It is also noted that the invention disclosed in this document is also useful for completely flat supports having different heights. Once the height differs from the reference height 29, the re-interpolation performs a scale correction.

The imaging system disclosed in this document may be applied, for example, to capture images at the output of a die cutting machine, where individual sheets tend to "fly" at a loosely controlled elevation.

Note that we always use "sheet" or "support" to denote the same object. If we want to emphasize the fact that the support is provided in sheet form, we mention "single sheet".