阅读说明：本技术 一种基于机器视觉的非接触式测脚方法 (Non-contact foot measuring method based on machine vision ) 是由 许源平 乔文博 张朝龙 许志杰 冯暄 蒋含 崔陆军 谢攀 于 2020-04-29 设计创作，主要内容包括：本发明涉及一种基于机器视觉的非接触式测脚方法,其包括相机标定和视觉测脚,相机标定用于确定测量区域和尺度信息并计算透射变换的参数；视觉测脚在相机标定获得的参数基础上,测量被测人的脚的测量值。本发明基于机器视觉,结合图像处理技术,通过提取脚的完整脚型轮廓实现脚长、脚宽的非接触式测量,不仅大大缩短了测量时间,提高了测量精度,并且具有较强的环境适应能力,可在不同光照环境下正常工作。(The invention relates to a non-contact foot measuring method based on machine vision, which comprises camera calibration and vision foot measuring, wherein the camera calibration is used for determining measurement area and scale information and calculating parameters of transmission transformation; the vision foot measuring device measures the measured value of the foot of the measured person on the basis of the parameters obtained by calibrating the camera. The invention is based on machine vision, combines with image processing technology, realizes the non-contact measurement of the foot length and the foot width by extracting the complete foot profile of the foot, not only greatly shortens the measurement time and improves the measurement precision, but also has stronger environmental adaptability and can normally work under different illumination environments.)

1. A non-contact foot measurement method based on machine vision is characterized in that: the method comprises camera calibration and visual foot measurement, wherein the camera calibration is used for determining a measurement area and scale information and calculating parameters of transmission transformation; the vision foot measuring device measures the measured value of the foot of the measured person on the basis of the parameters obtained by calibrating the camera.

2. The machine-vision-based non-contact foot measurement method of claim 1, wherein: the measured values of the feet of the tested person comprise a foot length and a foot width; the visual feet measurement needs to ensure that two feet of a measured person stand in parallel, and the abnormal situation can be identified when the two feet of the measured person stand in a shape like an inner Chinese character 'ba' or stand in a shape like an outer Chinese character 'ba' for more than 10 degrees; the visual foot measurement does not need to use a calibration plate, and the measured person stands on the pedal plate with two feet.

3. The non-contact foot measurement method based on machine vision as claimed in claim 1, wherein the main execution flow of the camera calibration is as follows:

(1.1) placing a calibration plate marked with a specific color on the pedal;

(1.2) shooting an image with a calibration plate by using a camera, converting the image from an RGB color space to an HSV color space, and extracting a green area of the calibration plate as a measurement area by using a color space threshold segmentation method;

(1.3) finding the outline of the green calibration plate by adopting an outline detection algorithm, extracting four corner points of the outline and the actual size of the calibration plate according to the outline, and calculating parameters and scale information of an image perspective transformation matrix;

and (1.4) saving the transmission transformation matrix parameters and scale information of the calibration plate image.

4. The machine-vision-based non-contact foot measurement method of claim 3, wherein: the contour detection algorithm in the step (1.3) adopts a FindContours () function in OpenCV.

5. The machine-vision-based non-contact foot measurement method of claim 3, wherein: in the step (1.1), the color of the pedal is red, and the calibration plate is green.

6. A non-contact pin measuring method according to claim 1 or 3, characterized in that: the camera calibration is only needed to be carried out once in the process of initializing the camera before measuring the foot, and calibration is not needed to be carried out again unless the camera deviates or the foot standing area changes.

7. The machine-vision-based non-contact foot measurement method of claim 1, wherein: the specific processing steps of the visual foot measurement are as follows:

(2.1) shooting a picture with the feet to be detected by using a camera, wherein the shot picture is an oblique view;

(2.2) performing transmission transformation on the picture shot in the step (2.1) according to parameters obtained by camera calibration, converting the picture into a top view so as to ensure that the pixel scale of the picture in the top view is uniform, and then calculating the values of the foot length and the foot width according to the ratio of the length and the width of the station foot region to the pixel points;

(2.3) extracting edge information of a foot from the image after transmission transformation by using a Canny operator, removing noise points beside the edge by using a Gaussian fuzzy algorithm, converting the image into a contour map of the foot, and setting a maximum threshold of the Canny operator to be 0 when extracting the edge information, namely, keeping all the edge information;

(2.4) extracting the outline of the foot, then extracting the minimum circumscribed rectangle of the outline, extracting the width of the left foot area and the right foot area and the distance between the toes of the two feet and the upper boundary of the areas, and judging various abnormal measuring conditions according to the information of the minimum circumscribed rectangle;

(2.5) through the processing of the above steps (2.1) - (2.4), a detection and distance calculation model can be obtained, taking the right foot as an example, wherein W is₁Representing the width of the foot-standing region, W₂To the width of the right foot to be measured, H₁Is the length of the foot standing region, H₂Distance from the toe of the right foot to the upper boundary of the region; c₁、C₂、D₁And D₂Representing the number of pixel points; due to W₁And H₁For a known quantity, and the pixel size is fixed, the calculation method of the foot width and the foot length can be obtained:

the length of the foot is:

H_foot＝H₁-H₂(3)。

8. the non-contact foot measuring method based on machine vision according to claim 7, wherein the step (2.4) of determining various abnormal measuring conditions specifically comprises:

firstly, two abnormal conditions of no foot placement and obstacles can be detected according to the detected number of the minimum circumscribed rectangles; if the minimum circumscribed rectangle is not detected, judging that the camera does not acquire an image with feet; if the detected minimum circumscribed rectangles are more than 2, judging that the acquired image has other obstacles similar to feet;

secondly, under the condition that the number of the minimum circumscribed rectangles is correct, the standing condition of the tested person at the moment can be judged according to the angle of the minimum circumscribed rectangle, and if the angle of the rectangle is within the range of 10-45 degrees, the feet of the tested person are judged to stand in an inner eight-posture; if the rectangular angle is in the range of-80 degrees to-45 degrees, the feet of the tested person are judged to stand in the eight-out postures.

9. The machine-vision-based non-contact foot measurement method according to claim 7, wherein in the step (2.4) the minimum bounding rectangle of the outline is extracted using the MinAreaRect () method of OpenCV.

Technical Field

The invention relates to a non-contact foot measuring method, in particular to a non-contact foot measuring method based on machine vision.

Background

With the development of times and social progress, the requirement of people on the comfort level of shoes and boots is higher and higher, in the last 60 th century, China carries out national foot type investigation twice, adopts a manual measurement method to measure and analyze the foot type of 25 tens of thousands of people, establishes shoe size standards and shoe tree samples suitable for Chinese people, traditional shoes and boots can be divided into men shoes, women's shoes and children's shoes according to gender and age, codes are divided according to the length of feet of people, and the difference between adjacent codes is 5 mm. However, the foot shape difference between foot length sections corresponding to the same size is very large, and even the left and right feet of the same person are not completely mirror-symmetrical, so that it is difficult to determine a proper shoe size simply according to the foot length index. In this case, many industrial production lines are shifting from large-scale batch-type production to custom-made, personalized, small-batch-type production. The shoe and boot personalized customization depends on accurate measurement of foot type characteristic data, the traditional foot type parameter measurement method mainly depends on manual measurement, the measurement method is low in efficiency, complex in process and high in labor cost, and different measurement personnel have certain difference in measurement size, so that the requirements of batch measurement, accurate measurement and rapid measurement cannot be met. In order to solve this problem, the foot shape measurement is gradually changed from a contact measurement method to a non-contact measurement method, wherein the non-contact measurement method mainly includes a line laser scanning method and a stereoscopic vision method.

The line laser scanning method has the advantages that the line laser scanning method has better three-dimensional measurement precision, but has some bottlenecks in application and popularization, mainly because the current three-dimensional foot type measurement equipment is expensive and is not beneficial to being used by small and medium-sized enterprises, and secondly, because the scanning time is longer, the user experience is not good; the method has the greatest characteristic of high shooting speed, can complete a shooting task in less than one second, and is suitable for occasions needing rapid measurement. However, the stereoscopic method requires matching of two images, and when the gray scale and the surface shape of the object do not change much, the matching and the measurement accuracy are easily affected. In actual life, only the data of the length and the width of the foot are needed to meet the requirement of the shoe type selection of the ordinary user, and other size parameters such as the arch height, the foot circumference, the length of the toes and the like are characteristic information of the foot type, but the data have almost no influence on the shoe type selection of the ordinary user. Therefore, aiming at the requirements of foot shape parameter measurement of current automation, low cost, high precision and simple and convenient operation, the invention realizes non-contact measurement of foot length and foot width by fully automatically extracting the complete foot shape outline of the foot based on a machine vision method and combining an image processing technology.

Disclosure of Invention

Aiming at the problems in the background technology, the invention provides a non-contact foot measuring method based on machine vision, which is combined with an image processing technology and extracts the complete foot profile of a foot to realize the non-contact measurement of the foot length and the foot width, thereby greatly shortening the measurement time, improving the measurement precision, having stronger environment adaptability and being capable of normally working under different illumination environments.

The technical scheme of the invention is as follows:

the non-contact foot measuring method based on the machine vision comprises camera calibration and vision foot measuring, wherein the camera calibration is used for determining information of a measuring area and scale and calculating parameters of transmission transformation; the vision foot measuring device measures the measured value of the foot of the measured person on the basis of the parameters obtained by calibrating the camera.

The non-contact foot measuring method based on machine vision comprises the following steps: the measured values of the feet of the tested person comprise a foot length and a foot width; the visual feet measurement needs to ensure that two feet of a measured person stand in parallel, and the abnormal situation can be identified when the two feet of the measured person stand in a shape like an inner Chinese character 'ba' or stand in a shape like an outer Chinese character 'ba' for more than 10 degrees; the visual foot measurement does not need to use a calibration plate, and the measured person stands on the pedal plate with two feet.

The non-contact foot measurement method based on the machine vision comprises the following main execution flows of camera calibration: (1.1) placing a calibration plate marked with a specific color on the pedal; (1.2) shooting an image with a calibration plate by using a camera, converting the image from an RGB color space to an HSV color space, and extracting a green area of the calibration plate as a measurement area by using a color space threshold segmentation method; (1.3) finding the outline of the green calibration plate by adopting an outline detection algorithm, extracting four corner points of the outline and the actual size of the calibration plate according to the outline, and calculating parameters and scale information of an image perspective transformation matrix; and (1.4) saving the transmission transformation matrix parameters and scale information of the calibration plate image.

The non-contact foot measuring method based on machine vision comprises the following steps: the contour detection algorithm in the step (1.3) adopts a FindContours () function in OpenCV.

The non-contact foot measuring method based on machine vision comprises the following steps: in the step (1.1), the color of the pedal is red, and the calibration plate is green.

The non-contact foot measuring method based on machine vision comprises the following steps: the camera calibration is only needed to be carried out once in the process of initializing the camera before measuring the foot, and calibration is not needed to be carried out again unless the camera deviates or the foot standing area changes.

The non-contact foot measuring method based on machine vision comprises the following steps: the specific processing steps of the visual foot measurement are as follows:

(2.1) shooting a picture with the feet to be detected by using a camera, wherein the shot picture is an oblique view;

(2.2) performing transmission transformation on the picture shot in the step (2.1) according to parameters obtained by camera calibration, converting the picture into a top view so as to ensure that the pixel scale of the picture in the top view is uniform, and then calculating the values of the foot length and the foot width according to the ratio of the length and the width of the station foot region to the pixel points;

(2.3) extracting edge information of a foot from the image after transmission transformation by using a Canny operator, removing noise points beside the edge by using a Gaussian fuzzy algorithm, converting the image into a contour map of the foot, and setting a maximum threshold of the Canny operator to be 0 when extracting the edge information, namely, keeping all the edge information;

(2.4) extracting the outline of the foot, then extracting the minimum circumscribed rectangle of the outline, extracting the width of the left foot area and the right foot area and the distance between the toes of the two feet and the upper boundary of the areas, and judging various abnormal measuring conditions according to the information of the minimum circumscribed rectangle;

(2.5) through the processing of the above steps (2.1) - (2.4), a detection and distance calculation model can be obtained, taking the right foot as an example, wherein W is₁Representing the width of the foot-standing region, W₂To the width of the right foot to be measured, H₁Is the length of the foot standing region, H₂Distance from the toe of the right foot to the upper boundary of the region; c₁、C₂、D₁And D₂Representing the number of pixel points; due to W₁And H₁For a known quantity, and the pixel size is fixed, the calculation method of the foot width and the foot length can be obtained:

the length of the foot is:

H_foot＝H₁-H₂(6)。

the non-contact foot measurement method based on machine vision, wherein the step (2.4) of judging various abnormal measurement conditions specifically comprises the following steps:

firstly, two abnormal conditions of no foot placement and obstacles can be detected according to the detected number of the minimum circumscribed rectangles; if the minimum circumscribed rectangle is not detected, judging that the camera does not acquire an image with feet; if the detected minimum circumscribed rectangles are more than 2, judging that the acquired image has other obstacles similar to feet;

secondly, under the condition that the number of the minimum circumscribed rectangles is correct, the standing condition of the tested person at the moment can be judged according to the angle of the minimum circumscribed rectangle, and if the angle of the rectangle is within the range of 10-45 degrees, the feet of the tested person are judged to stand in an inner eight-posture; if the rectangular angle is in the range of-80 degrees to-45 degrees, the feet of the tested person are judged to stand in the eight-out postures.

The non-contact foot measurement method based on machine vision is characterized in that in the step (2.4), a MinAreaRect () method of OpenCV is used for extracting the minimum bounding rectangle of the outline.

Has the advantages that:

the non-contact foot measurement method based on the machine vision is reasonable in concept and simple in operation process, realizes non-contact measurement of the foot length and the foot width by extracting the complete foot profile of the foot based on the machine vision and combining with an image processing technology, and can effectively meet the requirements of low-cost, high-efficiency and high-precision foot parameter measurement.

Compared with a line laser scanning and stereoscopic vision method, the non-contact foot measurement method based on the machine vision is based on the optical imaging technology, does not relate to complex algorithms such as three-dimensional matching and the like, and optimizes an image processing algorithm, so that the non-contact foot measurement method based on the machine vision not only greatly shortens the measurement time (the time for measuring the foot is 1 second), improves the measurement precision (the precision can reach 0.2mm), but also has stronger environment adaptability and can normally work under different illumination environments; in addition, the foot measuring device involved in the non-contact foot measuring method based on machine vision is simple in structure, and can measure only by one common camera and one pedal, so that the production cost is greatly reduced, and the popularization and the use of small and medium-sized enterprises are facilitated.

The invention relates to a non-contact foot measurement method based on machine vision, which comprises the steps of collecting an original image by using a single high-precision color camera and accurately extracting length and width information of a foot through an image processing algorithm; aiming at the defects of expensive equipment, lower measuring efficiency, unstable accuracy and weak environmental adaptability in the existing foot measuring technical scheme, the invention reduces the measuring cost and improves the measuring efficiency and accuracy. In addition, by adding the light source, the brightness of the image acquired every time can be ensured to be uniform and sufficient, and the environmental adaptability is improved.

Compared with the existing foot measuring method, the non-contact foot measuring method based on the machine vision greatly improves the measuring efficiency, ensures the measuring precision and reduces the measuring cost.

Drawings

FIG. 1 is a flow chart of camera calibration in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 2 is a diagram of an original image shot by a calibration plate in the step (1.2) of camera calibration in the non-contact foot measuring method based on machine vision;

FIG. 3 is a schematic diagram of the green color calibration plate area in step (1.2) of camera calibration in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 4 is a diagram of corner detection in the step (1.3) of camera calibration in the non-contact foot measuring method based on machine vision according to the present invention;

FIG. 5 is a flow chart of the vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 6 is a photograph of the original image taken with a foot in step (2.1) of the vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 7 is a top view of the machine vision-based non-contact foot measuring method of the present invention after perspective transformation in step (2.2) of the vision foot measuring;

FIG. 8 is a foot outline diagram of the step (2.3) of the vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 9 is a minimum circumscribed rectangle diagram in step (2.4) of the vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 10 is a schematic diagram of distance calculation in step (2.5) of the vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 11 is a flowchart of an implementation of a vision foot measurement in the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 12 is a low brightness footage diagram of a non-contact footage method based on machine vision according to an embodiment of the present invention;

FIG. 13 is a block diagram of a low brightness foot in one example of a non-contact foot measurement method based on machine vision according to the present invention;

FIG. 14 is a middle brightness foot-stand diagram of a second embodiment of the non-contact foot-measuring method based on machine vision according to the present invention;

FIG. 15 is a block diagram of a middle brightness foot type in a second embodiment of the non-contact foot measurement method based on machine vision according to the present invention;

FIG. 16 is a high brightness footage diagram of a third embodiment of the non-contact footage method based on machine vision according to the present invention;

FIG. 17 is a high brightness foot block diagram of a third embodiment of the non-contact foot measurement method based on machine vision according to the present invention;

fig. 18 is a schematic structural diagram of a conventional line laser foot type measuring instrument.

Detailed Description

The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, not all, embodiments of the present invention.

The non-contact foot measuring method based on machine vision comprises camera calibration and vision foot measuring, wherein the camera calibration is used for determining a measuring area and scale information, and meanwhile, as most foot measuring devices cannot be vertically provided with cameras, the camera calibration also needs to calculate parameters of transmission transformation; the vision measuring foot measures the measured value (including the foot length and the foot width) of the feet of the measured person on the basis of the parameters obtained by calibrating the camera, and in order to ensure the measurement precision, the vision measuring foot needs to ensure that the two feet of the measured person stand in parallel (the condition that the feet stand in the shape of an inner Chinese character 'ba' or the feet stand in the shape of an outer Chinese character 'ba' for more than 10 degrees can be identified as abnormal conditions); the image processing algorithm of the present invention may be implemented using OpenCV.

As shown in fig. 1, the main execution flow of the camera calibration is as follows:

(1.1) placing a calibration plate marked with a specific color on the pedal; in the invention, the color of the pedal is red, the calibration plate is green, and other distinguishable colors can be adopted;

(1.2) shooting an image with a calibration plate by using a camera (as shown in fig. 2), converting the image from an RGB (Red, Green, Blue) color space to an HSV (Hue, Saturation, Value) color space, and extracting a Green area of the calibration plate (as shown in fig. 3) by using a color space threshold segmentation method, wherein the area is a measurement area;

(1.3) finding the outline of the green calibration plate by adopting an outline detection algorithm (FindContours () function in OpenCV), extracting four corner points (shown in figure 4) of the outline and the actual size of the calibration plate according to the outline, and calculating parameters and scale information of an image perspective transformation matrix;

and (1.4) saving the transmission transformation matrix parameters and scale information.

The camera calibration is only needed to be carried out once in the process of initializing the camera before measuring the foot, and calibration is not needed to be carried out again unless the camera deviates or the foot standing area changes.

The visual foot measurement does not need to use a calibration plate, and the measured person stands on a red (other colors can be used) pedal with two feet;

as shown in fig. 5, the above-mentioned process of visual foot measurement mainly includes the following steps:

(2.1) using a camera to shoot a picture with the feet to be measured, wherein the shot picture is an oblique view as shown in figure 6.

(2.2) performing transmission transformation on the picture shot in the step (2.1) according to the parameters obtained by camera calibration, and converting the picture into a top view (as shown in fig. 7), so that the pixel scale of the picture in the top view can be ensured to be uniform, and the values of the foot length and the foot width (namely, the value of W in the step 2.5) can be calculated according to the ratio of the length and the width of the station foot region to the pixel point₂、H₂)。

(2.3) extracting edge information of a foot from the transmission-transformed picture by using a Canny operator, and removing noise points beside the edge by using Gaussian blur; converting the picture into a foot outline (as shown in fig. 8); when the edge information is extracted, the maximum threshold value of the Canny operator is set to be 0, namely all the edge information is reserved, and the accuracy of the edge information is effectively improved.

(2.4) after the outline of the foot is extracted, extracting the minimum bounding rectangle of the outline (using the MinAreaRect () method of OpenCV, the result is shown in FIG. 9); extracting the width of the left foot area, the width of the right foot area and the distance from the toes of the two feet to the upper boundary of the areas; and various abnormal measurement conditions can be judged through the information of the minimum circumscribed rectangle. The method specifically comprises the steps that firstly, according to the number of the detected minimum external rectangles, two abnormal conditions of no foot placement and obstacle existence can be detected; if the minimum circumscribed rectangle is not detected, judging that the camera does not acquire the images with the feet; if the detected minimum circumscribed rectangles are more than 2, judging that the acquired image has other obstacles similar to feet; secondly, under the condition that the minimum circumscribed rectangle is correct in number, the standing condition of the tested person at the moment can be judged according to the angle of the minimum circumscribed rectangle, and if the angle of the rectangle is within the range of (10 degrees and 45 degrees), the feet of the tested person are judged to stand in an inner eight-posture; if the rectangular angle is within the range of (-80 °, -45 °), the foot of the subject is determined to be standing in the eight-out posture.

(2.5) through the previous processing steps, a detection and distance calculation model (for the right foot as an example) can be obtained as shown in FIG. 10, where W is₁Representing the width of the foot-standing region, W₂To the width of the right foot to be measured, H₁Is the length of the foot standing region, H₂The distance from the toe of the right foot to the upper boundary of the area (because of the occlusion problem of the heel position, the measurement cannot be directly carried out); c₁、C₂、D₁And D₂Representing the number of pixel points; due to W₁And H₁Is a known quantity, and the dimension of the pixel is fixed, so the calculation mode of the foot width and the foot length can be obtained:

the length of the foot is:

H_foot＝H₁-H₂(9)

the foot measuring flow algorithm of the non-contact foot measuring method based on machine vision can use a Modbus communication protocol to communicate with external application software (an upper computer), and a specific implementation flow chart is shown in FIG. 11.

As shown in fig. 11, the implementation flow of the vision foot measurement in the non-contact foot measurement method based on machine vision of the present invention is divided into two parts:

(1) a camera calibration section: firstly, after receiving a calibration instruction sent by a Modbus, a pin measurement program calls a calibration algorithm in the pin measurement program, acquires a picture with a calibration board, judges whether calibration is successful according to the detected angular point position, and returns a reply instruction of whether calibration is successful to an upper computer;

(2) a foot measuring part: after the foot measuring program receives a foot measuring instruction sent by the Modbus, a foot measuring algorithm in the foot measuring program is called to control the camera to acquire images every 5 frames, 5 frames of images are acquired for measurement in total, results which do not meet requirements are deleted in the algorithm, and the average value of 5 correct measurement values is calculated and returned to the upper computer.

The non-contact foot measuring method based on machine vision can carry out normal measurement under different illumination conditions, and provides the following examples under three illumination conditions:

example one: under low light conditions, the person to be measured stands on both feet (see fig. 12), and the obtained foot shape is shown in fig. 13, and the measurement results are that the left foot is 229mm long, the left foot is 91mm wide, the right foot is 230mm long, and the right foot is 94mm wide.

Example two: under the condition of middle brightness, the human to be measured stands on both feet (as shown in FIG. 14), the obtained foot shape is shown in FIG. 15, and the measurement results are that the left foot length is 228mm, the left foot width is 91mm, the right foot length is 231mm and the right foot width is 94 mm.

Example three: under high brightness conditions, the person to be measured stands on both feet (see fig. 16), and the obtained foot shape is shown in fig. 17, and the measurement results are that the left foot is 228mm long, the left foot is 91mm wide, the right foot is 231mm long and the right foot is 94mm wide.

The invention is based on machine vision, combines with image processing technology, realizes the non-contact measurement of the foot length and the foot width by extracting the complete foot profile of the foot, not only greatly reduces the measurement cost, shortens the measurement time, improves the measurement precision, but also has stronger environmental adaptability and can normally work under different illumination environments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; while the invention has been described in terms of what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.