阅读说明：本技术 具有距离修正功能的测距装置 (Distance measuring device with distance correction function ) 是由 中村稔 高桥祐辉 渡边淳 于 2019-07-05 设计创作，主要内容包括：本发明提供一种具有距离修正功能的测距装置,具备：参照物体距离计算部,其根据二维图像来计算到参照物体的距离,该二维图像是拍摄了具有三维坐标的相关性明确的多个特征点的参照物体而得的图像；以及修正量计算部,其通过对计算出的到参照物体的距离和到距离图像中的参照物体的测距值进行比较,来计算用于修正距离图像的修正量。(The present invention provides a distance measuring device with a distance correction function, comprising: a reference object distance calculation unit that calculates a distance to a reference object from a two-dimensional image in which a reference object having a plurality of feature points with clear correlation in three-dimensional coordinates is captured; and a correction amount calculation unit that calculates a correction amount for correcting the distance image by comparing the calculated distance to the reference object with the distance measurement value to the reference object in the distance image.)

1. A distance measuring device comprising a light emitting unit that emits reference light to a measurement target space at a predetermined light emission timing, and a plurality of light receiving elements that are two-dimensionally arranged and receive incident light from the measurement target space at a predetermined imaging timing, and that output a distance image of an object to the measurement target space based on the amount of light received by the light receiving elements, and a two-dimensional image corresponding to the distance image,

the distance measuring device is provided with:

a reference object distance calculation unit that calculates a distance to the reference object from the two-dimensional image in which the reference object having a plurality of feature points with clear correlation in three-dimensional coordinates is captured; and

and a correction amount calculation unit that compares the calculated distance to the reference object with the distance value to the reference object in the distance image to calculate a correction amount for correcting the distance image.

2. The ranging apparatus as claimed in claim 1,

the reference object is a reference mark having a plurality of feature points whose correlation with the three-dimensional coordinates is known.

3. The ranging apparatus as claimed in claim 1,

the reference object is an arbitrary object, and the distance measuring device further includes a unit that indicates a feature amount of the object or a positional relationship between the plurality of objects as a correlation between the three-dimensional coordinates.

4. A ranging apparatus as claimed in any of claims 1 to 3 wherein,

the distance measuring device further includes: and a light emission/imaging timing control unit that controls the light emission timing or the imaging timing based on the correction amount.

5. A ranging apparatus as claimed in any of claims 1 to 3 wherein,

the distance measuring device further includes: and a distance image generating unit that generates the distance image based on the correction amount.

Technical Field

The present invention relates to a distance measuring device for measuring a distance to an object based on a flight time of light, and more particularly, to a distance measuring device having a distance correction function.

Background

As a distance measuring device that measures a distance to an object, a camera that outputs a time of flight (TOF) of the distance from a time of flight of light is known. In many TOF cameras, a phase difference method is employed in which measurement light intensity-modulated at a predetermined period is irradiated to a measurement target space, and a phase difference between the irradiated measurement light and reflected light from an object in the measurement target space is detected.

A TOF camera as a three-dimensional sensor may generate a distance measurement error due to variation in individual characteristics of electronic components (e.g., a light receiving element, an a/D conversion element, and the like), temporal change of the electronic components, and the like. The individual characteristics are not adjusted to within the target error by calibration performed by the camera provider under specific conditions at the time of shipment, but the distance measurement error may become large due to a difference in usage environment (particularly temperature), a temporal change, and the like. The degree of these ranging errors differs from individual to individual.

Japanese patent laid-open publication No. 2015-175752 discloses a distance image generating device using an optical time-of-flight distance image sensor. The distance image generation device determines parameters (size, coefficient, number of frames, and the like) of a filter used for smoothing an object having a known size, and smoothes the distance image.

Japanese patent laid-open publication No. 2015-056057 discloses a posture estimation method using a marker of a TOF camera. In the above-described posture estimating method, the mark is detected from the camera image, the mark region is cut out from the range image, the optimum plane is estimated from the cut-out range image, and the position and posture of the mark are estimated from the estimated plane.

Japanese patent application laid-open No. 2014-070936 discloses an error pixel detection apparatus that detects an error pixel of a TOF camera. The error pixel detection means compares a distance-corrected TOF image (luminance image) in which the TOF image is corrected based on the distance image with a captured image acquired from the capturing camera, and detects an error pixel including a measurement error in the distance image based on the comparison result.

Disclosure of Invention

Due to individual characteristics of the electronic elements of the TOF camera, particularly, semiconductor Laser (LD), transistors, resistors, and the like, timing delay and waveform blunting occur in the actual reference light with respect to the emission timing of the ideal reference light (see fig. 5). The timing delay and the waveform dullness are not random but repetitive, and can be regarded as a simple delay or an average delay for an ideal light emission pulse, and therefore, it is equivalent to a state in which the offset Δ t is simply added to the distance measurement value with respect to the phase of the reflected light, and it can be considered that the distance measurement value needs to be offset-corrected. Therefore, the provider of the TOF camera obtains, as a parameter at the time of calibration before shipment, a shift Δ L of the distance measurement value due to the shift Δ t of the individual difference for each TOF camera based on the result of distance measurement of the object located at the predetermined distance, and then outputs the distance measurement value corrected based on the shift Δ L inside the TOF camera.

However, when the TOF camera is actually used, the timing delay and waveform dullness (i.e., simple delay) corresponding to the offset Δ t vary due to a temperature change caused by ambient temperature, component heat generation, or the like, a temporal change in individual characteristics, or the like, and the distance measurement value also varies. That is, the range measurement value of each pixel in the TOF camera fluctuates. In order to cope with the above-described variation, attempts have been made to provide a temperature sensor inside the TOF camera and change the correction amount according to the detected temperature, but the correction cannot be performed accurately in accordance with the problems of the arrangement and accuracy of the temperature sensor.

Therefore, a distance measuring device capable of easily performing highly accurate distance correction is desired.

One aspect of the present disclosure provides a distance measuring device including a light emitting unit that emits reference light to a measurement target space at a predetermined light emission timing, and a plurality of light receiving elements that are two-dimensionally arranged and receive incident light from the measurement target space at a predetermined imaging timing, and outputting a distance image of an object in the measurement target space and a two-dimensional image corresponding to the distance image based on the light receiving amount of the light receiving elements. The distance measuring device is provided with: a reference object distance calculation unit that calculates a distance to a reference object from a two-dimensional image in which a reference object having a plurality of feature points with clear correlation in three-dimensional coordinates is captured; and a correction amount calculation unit that calculates a correction amount for correcting the distance image by comparing the calculated distance to the reference object and the distance measurement value of the reference object to the distance image.

Drawings

Fig. 1 is a block diagram of a ranging apparatus according to an embodiment.

Fig. 2A is an explanatory diagram illustrating a distance correction method using a reference mark as a reference object according to an embodiment.

Fig. 2B is an explanatory diagram illustrating a distance correction method using an arbitrary object as a reference object according to one embodiment.

Fig. 3 shows the distance measurement values at the time of installation of the distance measuring device of one embodiment, years later or when the ambient temperature changes greatly, and after correction.

Fig. 4 is an explanatory diagram illustrating a distance correction method according to another embodiment.

Fig. 5 shows a light emission timing delay and waveform blunting of the reference light of the related art.

Detailed Description

Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. The same or similar structural elements are denoted by the same or similar reference symbols in the respective drawings. The embodiments described below do not limit the technical scope of the invention described in the claims and the meaning of the technical terms.

Fig. 1 is a block diagram of a distance measuring device 10 according to the present embodiment. The distance measuring device 10 is a TOF camera that measures a distance to an object, for example, according to a phase difference method, and includes: a light emitting unit 11 that emits reference light L1 irradiated into a measurement target space; a light receiving unit 12 that receives incident light L2 from the measurement target space; and a distance image generating unit 13 that generates a distance image of the object in the measurement target space.

The light emitting unit 11 is configured by a light source such as a Light Emitting Diode (LED) or an LD that emits Near Infrared (NIR) light, and emits reference light L1 intensity-modulated at a predetermined cycle in accordance with a light emission timing signal from the light emission/imaging timing control unit 14. The reference light L1 is diffused by the diffusion plate 15 and irradiated to the measurement target space.

The light receiving unit 12 is configured by an image sensor such as a CCD or CMOS having an RGB filter, an NIR filter, or the like, for example, and receives incident light L2 via an optical system 16 including a condenser lens or the like. The incident light L2 includes external light in addition to the reference light reflected by the object. The light receiving unit 12 includes 4 light receiving elements 17 for receiving red light, blue light, green light, and NIR light for one pixel. Alternatively, the light receiving unit 12 has one light receiving element that receives only NIR light for one pixel.

The light receiving element 17 is formed of, for example, a photodiode, a capacitor, or the like. The light receiving element 17 that receives the NIR light receives light at a plurality of imaging timings delayed by a predetermined phase from the light emission timing of the reference light L1, based on the imaging timing signal from the light emission/imaging timing control unit 14. For example, as shown in fig. 5, the received light amount Q is acquired at 4 imaging timings Et1 to Et4 shifted by phases of 0 °, 90 °, 180 °, and 270 ° from the light emission timing of the ideal reference light₁～Q₄. On the other hand, the light receiving elements 17 that receive red light, blue light, and green light acquire the light receiving amounts in a predetermined imaging period. As shown in fig. 1, the acquired light receiving amount is amplified by the amplifying section 18, a/D converted by the a/D converting section 19, and the a/D converted value is stored in the buffer memory 20.

The distance image generating unit 13 receives the light quantity Q from the NIR light₁～Q₄Generates a distance image 30 to the object of the measurement target space. Distance measurement value L_tofFor example, by the well-known following formula. Where Td is the phase difference between the reference light and the reflected light, c is the speed of light, and f is the frequency. The generated distance image is stored in the buffer memory 21 and output to an application (application software app)23 via the output control unit 22.

The two-dimensional image generation unit 24 generates a two-dimensional image 31 from the a/D conversion value of the light receiving amount of RGB light or NIR light. That is, the two-dimensional image 31 may be an RGB image (color image) or an NIR image (monochrome image). The two-dimensional image 31 is stored in the buffer memory 21 and output to the application 23 via the output control unit 22.

Referring to fig. 5, as described above, the light emission timing of the actual reference light has the deviation Δ t from the light emission timing of the ideal reference light, the deviation Δ L of the distance measurement value by the deviation Δ t is acquired by calibration performed at the time of shipment, and the distance measurement value corrected by the deviation Δ L is output inside the TOF camera. Thus, the distance measurement value L_tofThe calculation is performed by the following equation after correction with the offset Δ L added.

Since the offset Δ t varies due to temperature change, time change, and the like, the final range image may include a range error. The distance measuring device 10 of the present embodiment has a distance correction function for correcting the variation amount of the offset Δ t (further, for correcting the distance image). The distance measuring apparatus 10 calculates a correction amount Li for correcting the distance image 30 using the distance to the reference object 25 (see fig. 1) geometrically calculated from the two-dimensional image 31. In order to geometrically calculate the distance to the reference object 25, the reference object 25 needs to have a plurality of feature points 32 whose correlation with the three-dimensional coordinates is clear. The correlation between three-dimensional coordinates is clearly understood to mean that a relative positional relationship can be known. That is, it is not necessarily required that the correlation of the three-dimensional coordinates be known (i.e., the ranging apparatus 10 need not be stored in a memory or the like in advance).

Fig. 2A is an explanatory diagram illustrating a distance correction method using a reference mark 25a as the reference object 25. As shown in fig. 4, the reference mark 25a is formed by arranging a regular circle, a square, and a rhombus having a known positional relationship on a white plate-like member having a quadrangular shape with a known size, and includes a large number of feature points having a known correlation in three-dimensional coordinates. For example, the feature points may be the centers of circles, squares, and diamonds (represented by the symbols 32a, 32b, 32 c). Further, the perfect circular center portion of the reference mark 25a is set as the representative feature point 32 b. The distance measuring device 10 detects the reference mark 25a from a two-dimensional image obtained by imaging the reference mark 25a by known image processing, and specifies the position coordinates of various feature points of the reference mark 25a on the image at a sub-pixel level (level).

The distance measuring device 10 geometrically calculates the distance L to the representative feature point 32b based on a combination of position coordinates of a plurality of (generally 4 or more) feature points on the image_ref. To calculate a distance L with higher accuracy_refCalculating a plurality of L from a plurality of feature points different in combination_refAveraging processing is performed. The distance measuring device 10 measures the distance L to the representative feature point 32b calculated from the two-dimensional image_refAnd a distance measurement value L of the representative feature point 32b of the range image_tofA comparison is made to calculate a correction amount Li for correcting the distance image. In the distance correction method described above, the two-dimensional image and the range image correspond to each pixel, and when the position coordinates of the representative feature point 32b on the image are determined by the sub-pixel level without the need of the process of associating or merging 2 image feature points as in the known stereo method, the distance measurement value of the representative feature point on the range image can be calculated with high accuracy by the interpolation process of the distance measurement values with the surrounding pixels, and therefore, the correction amount Li with high accuracy can be calculated. In addition, the reference mark 25a is prepared in advance, so that the user can easily perform the correction work when the correction is to be performed. Alternatively, the distance measuring device 10 may always take an image of the reference mark 25a and maintain the accuracy by changing the correction amount Li as needed.

Fig. 2B is an explanatory diagram illustrating a distance correction method using arbitrary objects 25B and 25c as reference objects. This example is an example in which the three-dimensional coordinate correlations of 10 feature points of 2 cuboids 25b, 25c are indicated by 9 vectors. For example, the operator may directly input three-dimensional coordinates of 9 vectors into the distance measuring device 10, or may indicate the positions of 8 corners 32d, 32e, 32f, 32g, 32h, 32i, 32j, 32k of 2 cuboids on a two-dimensional image taken by the distance measuring device 10, and input the length of each side of the 2 cuboids, the distance between the 2 cuboids. In this way, the distance measuring device 10 preferably includes a means for indicating the feature amount of the object (for example, three-dimensional coordinates of a vector, the position of a feature point of the object, the size of the object itself, the specific shape of the object, the size of a pattern or a color, etc.) or the positional relationship of a plurality of objects (for example, the distance between objects, etc.) as the correlation of the three-dimensional coordinates. Thus, any of the objects 25b, 25c will function in the same manner as the reference mark 25 a.

Referring again to fig. 1, the distance measuring device 10 of the present embodiment includes a reference object distance calculating unit 26 that calculates a distance to the reference object 25, and a correction amount calculating unit 27 that calculates a correction amount Li for correcting the distance image 30 using the distance to the reference object 25. The reference object distance calculation unit 26 and the correction amount calculation unit 27 can be configured as software that causes a processor such as a Central Processing Unit (CPU) to function, for example. Or may be implemented in hardware, for example, as a processor capable of executing processing of at least a part of the software.

The reference object distance calculation unit 26 reads out the two-dimensional image 31 from the buffer memory 21, and calculates the distance to the feature point 32 geometrically from the two-dimensional image 31, the two-dimensional image 31 being an image obtained by imaging the reference object 25 having a plurality of feature points 32 (including the representative feature point 32b) whose correlation with the three-dimensional coordinates is clear.

The correction amount calculation unit 27 calculates the distance L to the representative feature point 32b by the reference object distance calculation unit 26_refAnd the range value L of the representative feature point 32b of the range image 30 stored in the buffer memory 21_tofAnd comparing to calculate a correction amount for correcting the distance image. For example, the correction amount Li may be represented by the following formula as the distance L_refAnd a distance measurement value L_tofThe value calculated by the difference value of (a), or the value of a plurality of coefficient value groups in a functional expression obtained by a separate verification test or the like in which a higher correction is performed for various distance measurement values of all pixels.

Li＝L_tof-L_ref

The correction amount calculation unit 27 stores the correction amount Li in the nonvolatile memory 28, and reads out the correction amount Li from the nonvolatile memory 28 and reuses it when the power supply of the distance measuring device 10 is turned on. Alternatively, in the method of using the distance measuring device 10 to always capture the reference mark 25a, the correction amount Li may be changed as needed to maintain the accuracy. The correction amount calculation unit 27 outputs the correction amount to the light emission/capturing timing control unit 14 or the distance image generation unit 13.

The light emission/imaging timing control unit 14 controls the light emission timing or the imaging timing based on the correction amount Li. For example, when the correction amount Li is a correction value of the distance as in the above equation, the light emission/imaging timing control unit 14 calculates the corrected shift Δ t 'according to the following equation, and shifts the imaging timing or the light emission timing so as to delay the imaging timing by the shift Δ t'. In addition, 2 times is a distance by which the reflected light moves by 2 times the distance measurement value.

Alternatively, the distance image generating unit 13 may correct the distance image according to the correction amount Li. For example, when the correction amount Li from the correction amount calculation unit 27 is valid, the distance image generation unit 13 corrects the distance measurement value L by superimposing the correction amount Li in addition to the correction of the offset Δ L as shown in the following formula_tof。

Fig. 3 shows the distance measurement values at the time of installation of the distance measuring device 10, years later, or when the ambient temperature has changed greatly, and after correction. When the distance measuring device 10 is installed, the distance measuring value is within the allowable range, but after years or when the ambient temperature changes greatly, the distance measuring value deviates from the allowable range, and therefore the operator corrects the distance measuring value using the distance correction function of the distance measuring device 10. Alternatively, the distance measuring device 10 may continuously monitor the reference object 25, and perform distance correction as needed to maintain accuracy. According to the distance measuring device 10, it is possible to easily correct a distance measuring error caused by variation in individual characteristics of electronic components, temporal variation of electronic components, or the like.

As one of the distance correction methods according to the other embodiments of the present application, a method of using a plurality of distance values on the plane of the reference object in addition to the previously described method of using the distance representing the feature point will be described with reference to fig. 4. The distance measuring apparatus 10 measures the distance based on TOF, and therefore, unlike the known stereo method, can also measure the distance of an object plane having no light intensity variation. Therefore, the reference object distance calculation unit 26 and the correction amount calculation unit 27 can calculate the position and orientation of the plane 33 from a plurality of feature points of the two-dimensional image obtained by imaging the plane 33, and can calculate the correction amount Li for correcting the distance image by comprehensively comparing the distance value of each pixel of the imaged plane 33 with the distance value of each pixel on the distance image corresponding to the pixel. Thus, the distance measuring device 10 performs correction with higher accuracy. The position of the plane 33 indicates the distance value of the plurality of feature points, and the posture of the plane 33 indicates the inclination of the plane 33 with respect to the optical axis of the distance measuring device 10.

For example, the reference object distance calculating unit 26 detects the reference mark 25a and a plurality of feature points (for example, the center portions of 4 corners, a perfect circle, a square, and a rhombus) with respect to the reference mark 25a including 4 corners 32l, 32m, 32n, and 32o, for example, from the two-dimensional image, and obtains the position and orientation of the plane 33. Next, the reference object distance calculation unit 26 calculates a distance value for each pixel for specifying the plane 33 from the position and orientation of the plane 33, and outputs the position on the image of each pixel and the distance value for each pixel after the specification to the correction amount calculation unit 27. The correction amount calculation unit 27 calculates a difference between the average value of the distance values of the respective pixels and the average value of the respective distance measurement values on the distance image corresponding to the determined positions of the respective pixels, and calculates the correction amount Li.

In a distance image based on the TOF principle, the accuracy of a pixel having strong light intensity is generally higher than that of a pixel having weak light intensity. Therefore, when averaging the respective distance measurement values on the range image corresponding to the positions on the image of the respective pixels specified as described above, weighted averaging after weighting the light intensities of the respective pixels can be performed. Thus, the correction amount Li can be obtained with higher accuracy. The light intensity I is calculated, for example, according to the well-known following formula.

According to the above embodiment, since the two-dimensional image 31 and the distance image 30 are associated with each other for each pixel, it is possible to calculate a correction amount for correcting the distance image 30 using the distance to the reference object 25, the distance being geometrically calculated from the two-dimensional image 31. As a result, it is possible to easily correct a distance measurement error caused by variation in individual characteristics of the electronic components, temporal change of the electronic components, and the like.

While various embodiments have been described in the present specification, the present invention is not limited to the above embodiments, and it is desirable to recognize that various modifications can be made within the scope described in the claims.