阅读说明：本技术 手部关节定位及局部区域面积计算方法、处理器和存储器 (Hand joint positioning and local area calculation method, processor and memory ) 是由 陈虎 涂燕晖 陈一昕 于 2021-08-31 设计创作，主要内容包括：本申请公开了一种手部关节定位和局部区域面积的计算方法,通过获得手部的点位数据,所述点位数据包括坐标数据和颜色数据,所述点位数据通过扫描带有手部点位标识的手部图像获得；所述上述手部点位标识位于分割手部结构为远节、中节、近节以及手掌的关节的两端,形成按照手部结构划分的远节点位、中节点位、近节点位以及手掌点位4种点位位置类型；将每个点位的中心绕旋转点旋转成手部竖直状态,所述旋转点为形成手掌的各个手部点位的质心；根据点位位置类型以及点位中心的坐标完成手部结构对应位置的定位。通过本申请解决了对手部关节进行精确定位和局部区域面积检测的问题,从而能够对手部进行更为精确的后续研究。(The application discloses a hand joint positioning and local area calculating method, which comprises the steps of obtaining point location data of a hand, wherein the point location data comprises coordinate data and color data, and the point location data is obtained by scanning a hand image with a hand point location mark; the hand point location marks are positioned at two ends of joints of which the hand structures are far nodes, middle nodes, near nodes and palms, and 4 point location types of the far nodes, the middle nodes, the near nodes and the palms are formed according to the division of the hand structures; rotating the center of each point position around a rotating point to be in a hand vertical state, wherein the rotating point is the mass center of each hand point position forming a palm; and finishing the positioning of the corresponding position of the hand structure according to the type of the point position and the coordinates of the center of the point position. Through the method and the device, the problems of accurate positioning and local area detection of the hand joints are solved, so that more accurate follow-up research can be carried out on the hand.)

1. A hand joint positioning method is characterized in that:

acquiring point location data of a hand, wherein the point location data comprises coordinate data and color data, and the point location data is acquired by scanning a hand image with a hand point location mark; the hand point location marks are positioned on the hand contour line and positioned at two ends of joints dividing the hand structure into a far node, a middle node, a near node and a palm, and 4 point location types of the far node location, the middle node location, the near node location and the palm location are formed according to the division of the hand structure;

rotating the center of each point position around a rotating point to form a hand vertical state, wherein the rotating point is the mass center of each hand point position forming a palm, and the hand vertical state is based on the middle section vertical of the index finger;

and finishing the positioning of the corresponding position of the hand structure according to the type of the point position and the coordinates of the center of the point position.

2. The method of claim 1, wherein: the hand point location identification is obtained by marking the 3D drawing software of Windows10, and each point location type corresponds to being marked with different colors in the 3D drawing software.

3. The method of claim 2, wherein: the center of each point position is rotated to be in a hand vertical state around a rotating point, and the operation is carried out according to the following mode:

acquiring the image coordinate of the rightmost end of the bottom edge of each point location, the length w of the bottom edge of the point location and a scale h, calculating the image coordinate of the center of the point location, and converting the image coordinate of the center of the point location into Cartesian coordinates (x, y);

acquiring 4 point locations at two joints corresponding to a middle section forming the index finger, connecting every two point locations corresponding to each joint to obtain a connecting line of midpoints of the two connecting lines, and obtaining an included angle alpha between the connecting line and a vertical line;

the coordinates (X, Y) of the center of the rotated point location are calculated as follows

X＝(x-x₀)*cos(a)-(y-y₀)*sin(a)+x₀，

Y＝(x-x₀)*sin(a)-(y-y₀)*cos(a)+y₀，

Wherein (x)₀,y₀) Is the cartesian coordinates of the rotation point.

4. The method of claim 3, wherein: the obtaining of the coordinate (x, y) of the rightmost end of the bottom edge of each point location is performed as follows:

firstly, acquiring all point location coordinate sets of a certain color;

secondly, sorting all point positions in the set according to the size of a vertical coordinate;

thirdly, positioning w continuous coordinate points at the end part of the larger longitudinal coordinate end of the queue according to the queuing sequence, wherein the point with the largest horizontal coordinate in the w coordinate points is the coordinate of the rightmost end of the bottom edge of the point position to which the w coordinate points belong, and defining the position as the starting point of the point position to which the w coordinate points belong;

fourthly, obtaining all coordinate points corresponding to the point positions to which the w coordinate points belong to form current point position coordinates, removing the current point position coordinates from the original point position coordinate set to form a new set, and replacing the original point position coordinate set;

and repeating the second step to the fourth step until the coordinates of the rightmost end of the bottom edges of all the point positions of the color are determined.

5. The method of claim 4, wherein: the method for obtaining all coordinate points corresponding to the point locations to which the w coordinate points belong comprises the following steps:

defining an entry point P0 and an upward movement determination point F _ U as parameters associated with a row, where the entry point P0 represents an initial determination point as to whether a coordinate point of the row in which the entry point P0 is located belongs to a corresponding point, and the upward movement determination point F _ U is used to determine whether the entry point P0 moves upward for updating; the initial entry point P0 is a starting point of the point to which the w coordinate points belong, and the initial upward movement judgment point F _ U is another point in the same row as the initial entry point P0;

when judging for each current line, the following processes are carried out:

judging whether the entry point P0 belongs to a target point, and respectively judging whether the adjacent coordinate points of the line where the entry point P0 belongs are the current color or not one by one leftwards and rightwards by taking the position of the entry point P0 as a starting point until the judgment of all the coordinate points of the current line is finished;

in the judging process, if a certain coordinate point of the current row is the current color, judging that the coordinate point belongs to the target point position, otherwise, finishing the direction judgment of the current row; when the coordinate point belongs to the target point position, further judging whether a coordinate point above the coordinate point is a current color or not, if so, taking the coordinate point above the coordinate point as an entry point P0 of a previous line of the current line and continuing to judge the coordinate points of the current line one by one, and if not, directly continuing to judge the coordinate points of the current line one by one;

when the left and right directions of the current row are judged to be finished, if the current entering point P0 is in the current color at the moment and the current entering point P0 is different from the upward moving judgment point F _ U, updating the upward moving judgment point F _ U by using the coordinate position of the current entering point P0, and starting to judge a new row by using the current entering point P0 as the judgment starting point of the row; if the above condition is not satisfied, the judgment is finished.

6. The method of claim 1, wherein: the method for positioning the corresponding position of the hand structure according to the type of the point location and the coordinates of the point location center comprises the following steps:

obtaining all point location centers corresponding to a certain hand structure, and obtaining the centroid position (x) of all the point location centers₀,y₀)；

The angular position β of each point location center (x, y) relative to the centroid is calculated as follows:

and determining the relative position relation between the centers of the point positions according to the angle position beta, and determining the corresponding relation between the hand structure and each point position according to the relative position relation to complete positioning.

7. A method for calculating the area of a local area of a hand is characterized in that: a method of hand joint positioning according to any of claims 1-6, obtaining a convex polygon composed of points belonging to a certain hand structure, and calculating the area of said certain hand structure according to the following formula:

wherein P is a pixel of the convex polygon,is the area-to-pixel ratio of the hand image.

8. The method of claim 7, wherein: when the certain hand structure is a distal joint position, the

9. A processor for executing software, characterized in that the software is adapted to perform the method of any of claims 1 to 8.

10. A memory for storing software, characterized in that the software is adapted to perform the method of any of claims 1 to 8.

Technical Field

The application relates to the field of measurement, in particular to a finger joint positioning and area obtaining method and device.

Background

In some application scenarios, it is necessary to precisely locate joints of a hand and obtain the area of a hand region composed of adjacent joints, so as to obtain subsequent data related to the positions of the joints of the hand and the area of the corresponding hand region.

With the advent of neural network algorithms, algorithms for fully automatically implementing image target detection have existed. However, in the prior art, the neural network algorithm for calculating relevant data of the hand by fully automatically realizing image target detection is mainly focused on the gesture, for example, the google MediaPipe horizon provides a uniform topology for 540 key points (33 gestures, 21 hands and 468 individual face key points) and realizes near real-time performance on a mobile device. The application aims to realize integral and synchronous perception of body language, gestures and facial expressions and finish man-machine interaction. However, the number of gesture key points is too small to meet the requirement of 45 key point positioning at 15 positions.

Because there is no supervised large-sample-size data set of 45 key points of 15 joint parts of the hand at home and abroad, the full-automatic target detection algorithm based on the neural network cannot realize accurate positioning and area detection of the hand at present.

Disclosure of Invention

The embodiment of the application provides a hand joint positioning and local area calculating method, which is used for at least solving the problem that the hand joint cannot be accurately positioned and the local area cannot be detected in the prior art.

According to one aspect of the application, a hand joint positioning method is provided, point location data of a hand is obtained, the point location data comprises coordinate data and color data, and the point location data is obtained by scanning a hand image with hand point location marks; the hand point location marks are positioned on the hand contour line and positioned at two ends of joints dividing the hand structure into a far node, a middle node, a near node and a palm, and 4 point location types of the far node location, the middle node location, the near node location and the palm location are formed according to the division of the hand structure;

rotating the center of each point position around a rotating point to form a hand vertical state, wherein the rotating point is the mass center of each hand point position forming a palm, and the hand vertical state is based on the middle section vertical of the index finger;

and finishing the positioning of the corresponding position of the hand structure according to the type of the point position and the coordinates of the center of the point position.

Further, in the present invention, the hand point location marks are obtained by marking in 3D drawing software of Windows10, and each point location type corresponds to being marked with a different color in the 3D drawing software.

Further, in the present invention, the center of each point position is turned into a hand-vertical state around the rotation point, and the process is performed as follows:

acquiring the image coordinate of the rightmost end of the bottom edge of each point location, the length w of the bottom edge of the point location and a scale h, calculating the image coordinate of the center of the point location, and converting the image coordinate of the center of the point location into Cartesian coordinates (x, y);

acquiring 4 point locations at two joints corresponding to a middle section forming the index finger, connecting every two point locations corresponding to each joint to obtain a connecting line of midpoints of the two connecting lines, and obtaining an included angle alpha between the connecting line and a vertical line;

the coordinates (X, Y) of the center of the rotated point location are calculated as follows

X＝(x-x₀)*cos(a)-(y-y₀)*sin(a)+x₀，

Y＝(x-x₀)*sin(a)-(y-y₀)*cos(a)+y₀，

Wherein (x)₀,y₀) Is the cartesian coordinates of the rotation point.

Further, in the present invention, the obtaining of the coordinate (x, y) of the rightmost end of the bottom side of each point location is performed as follows:

firstly, acquiring all point location coordinate sets of a certain color;

secondly, sorting all point positions in the set according to the size of a vertical coordinate;

thirdly, positioning w continuous coordinate points at the end part of the end part with the larger vertical coordinate of the queue according to the queuing sequence, wherein the point with the largest horizontal coordinate in the w coordinate points is the coordinate at the rightmost end of the bottom edge of the point position to which the w coordinate points belong;

fourthly, obtaining all coordinate points corresponding to the point positions to which the w coordinate points belong to form current point position coordinates, removing the current point position coordinates from the original point position coordinate set to form a new set, and replacing the original point position coordinate set;

and repeating the second step to the fourth step until the coordinates of the rightmost end of the bottom edges of all the point positions of the color are determined.

Further, in the present invention, the method for obtaining all coordinate points corresponding to the point locations to which the w coordinate points belong includes:

defining an entry point P0 and an upward movement determination point F _ U as parameters associated with a row, where the entry point P0 represents an initial determination point as to whether a coordinate point of the row in which the entry point P0 is located belongs to a corresponding point, and the upward movement determination point F _ U is used to determine whether the entry point P0 moves upward for updating; the initial entry point P0 is a starting point of the point to which the w coordinate points belong, and the initial upward movement judgment point F _ U is another point in the same row as the initial entry point P0;

when judging for each current line, the following processes are carried out:

judging whether the entry point P0 belongs to a target point, and respectively judging whether the adjacent coordinate points of the line where the entry point P0 belongs are the current color or not one by one leftwards and rightwards by taking the position of the entry point P0 as a starting point until the judgment of all the coordinate points of the current line is finished;

in the judging process, if a certain coordinate point of the current row is the current color, judging that the coordinate point belongs to the target point position, otherwise, finishing the direction judgment of the current row; when the coordinate point belongs to the target point position, further judging whether a coordinate point above the coordinate point is a current color or not, if so, taking the coordinate point above the coordinate point as an entry point P0 of a previous line of the current line and continuing to judge the coordinate points of the current line one by one, and if not, directly continuing to judge the coordinate points of the current line one by one;

when the left and right directions of the current row are judged to be finished, if the current entering point P0 is in the current color at the moment and the current entering point P0 is different from the upward moving judgment point F _ U, updating the upward moving judgment point F _ U by using the coordinate position of the current entering point P0, and starting to judge a new row by using the current entering point P0 as the judgment starting point of the row; if the above condition is not satisfied, the judgment is finished.

Further, in the present invention, the method for completing the positioning of the corresponding position of the hand structure according to the type of the point location and the coordinates of the center of the point location is as follows:

obtaining all point location centers corresponding to a certain hand structure, and obtaining the centroid position (x) of all the point location centers₀,y₀)；

The angular position β of each point location center (x, y) relative to the centroid is calculated as follows:

and determining the relative position relation between the centers of the point positions according to the angle position beta, and determining the corresponding relation between the hand structure and each point position according to the relative position relation to complete positioning.

According to a second aspect of the present invention, there is provided a method for calculating an area of a local area of a hand, wherein a convex polygon composed of points belonging to a certain hand structure is obtained by the method for positioning hand joints, and the area of the certain hand structure is calculated according to the following formula:

wherein P is a pixel of the convex polygon,is the area-to-pixel ratio of the hand image.

Further, in the present invention, when the certain hand structure is a distal joint positionSaid

According to a third aspect of the present application, there is provided a processor for executing software for performing the above method.

According to a fourth aspect of the present application, there is provided a memory for storing software for performing the above method.

In the embodiment of the application, a hand joint positioning method is adopted to obtain point location data of a hand, wherein the point location data comprises coordinate data and color data, and the point location data is obtained by scanning a hand image with a hand point location mark; the hand point location marks are positioned on the hand contour line and positioned at two ends of joints dividing the hand structure into a far node, a middle node, a near node and a palm, and 4 point location types of the far node location, the middle node location, the near node location and the palm location are formed according to the division of the hand structure; rotating the center of each point position around a rotating point to form a hand vertical state, wherein the rotating point is the mass center of each hand point position forming a palm, and the hand vertical state is based on the middle section vertical of the index finger; completing the positioning of the corresponding position of the hand structure according to the type of the point location and the coordinates of the center of the point location; and further converting the area of the local area according to the proportion of the pixels of the local area occupying the pixels of the whole hand through a convex polygon formed by all point positions related to a certain hand structure. Through the method and the device, the problems of accurate positioning and local area detection of the hand joints are solved, so that more accurate follow-up research can be carried out on the hand.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

FIG. 1 is a flow chart of a hand joint positioning method according to an embodiment of the present application;

FIG. 2 is a schematic illustration of 45 points of a hand according to an embodiment of the present application;

FIG. 3 is a schematic diagram of the shape of the marking points corresponding to different drawing scales used in the embodiment of the present application;

FIG. 4 is a drawing scale h corresponding to a bottom side length w of a detection point in the embodiment of the present application;

FIG. 5 is a schematic view of the center of an identification point on scale 15 in an embodiment of the present application;

FIG. 6 is a flow chart of determining centers of all inspection points for a certain color in the embodiment of the present application;

FIG. 7 is a schematic diagram of single detection point pixel clustering in the embodiment of the present application;

fig. 8 is a flowchart of single detected point pixel clustering in the embodiment of the present application.

Fig. 9 is a flowchart of a reverse sorting algorithm for combining points in the embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

Because there is no supervised large-sample-size data set of 45 key points of 15 joint parts of the hand at home and abroad, the full-automatic target detection algorithm based on the neural network cannot realize accurate positioning of the hand at present. Therefore, a hand joint positioning calculation method as shown in fig. 1 is proposed to complete the accurate positioning of 45 key points at 15 parts of the hand.

As shown in fig. 1, a hand joint positioning method includes the following steps:

s102, point location data of the hand are obtained, wherein the point location data comprise coordinate data and color data, and the point location data are obtained by scanning a hand image with a hand point location mark; the hand point location marks are positioned on the hand contour line and positioned at two ends of joints dividing the hand structure into a far node, a middle node, a near node and a palm, and 4 point location types of the far node location, the middle node location, the near node location and the palm location are formed according to the division of the hand structure;

s104, rotating the center of each point position around a rotating point to form a hand vertical state, wherein the rotating point is the center of mass of each hand point position forming a palm, and the hand vertical state is based on the middle section vertical of the index finger;

and S106, positioning the corresponding position of the hand structure according to the type of the point position and the coordinates of the point position center.

The method combines two main processes of manual marking point position identification and automatic positioning, and is a semi-automatic detection algorithm for positioning the key joints of the hands. The method comprises the steps of marking key points of joints and palm areas on a hand image manually and reading to form point location data, wherein each point location data has parameters of two dimensions of coordinates and colors, the coordinates represent specific positions of the point locations, the colors represent approximate areas of the point locations, the process of turning hands into a vertical state by rotating the central coordinates of the point locations is simplified, the vertical state of the hands is beneficial to better utilizing the coordinates and the colors of the point locations to distinguish the point locations, accordingly, the datamation point locations and the specific structures of the hands can form corresponding relations, and finally, the structure location of the hands is completed.

In the embodiment of the present invention, the following hardware and software configurations are specifically performed:

hardware configuration aspect: a cell phone, camera or image scanner with over 800 million pixels; the hand fixed distance support is matched with a mobile phone, a camera or a scanner for use.

Software aspect: 3D drawing software for Windows 10. The integrated development environment for image detection on the local or cloud comprises ubuntu13.10, docker, python3.6, flash 1.0.2, gunicorn19.9.0, geventt 1.4.0 and opencv-4.4.0.46, supporting high concurrency requests with a process number of 30.

As the 3D drawing carried by the Windows10 can be used for making a three-dimensional model and drawing, the drawing of detection points with the size of 1-100 pixels can be completed. Therefore, in the embodiment of the invention, the hand point location identification is obtained by marking through 3D drawing software of Windows10, and each point location type corresponds to being marked with different colors in the 3D drawing software.

As shown in fig. 2, the hand point location marks are obtained by taking a picture of a hand by a camera and marking the key positions of the hand picture by a 3D drawing carried by Windows 10. In the figure, the English letters RGBC represent different colors selected for marking, wherein R represents red, G represents green, B represents blue, and C represents cyan. The number following the letter indicates the serial number of the logo in that color.

Table 1 shows the correspondence between the finger palm name and the identification point.

TABLE 1

Finger and palm names	Key point combination
		Thumb distant joint	(R9,R9|10,R10,G10,G9)
Proximal joint of thumb	(G9,G10,C10,C9)
		Distal section of forefinger	(R7,R7|8,R8,G8,G7)
Middle section of index finger	(G7,G8,B8,B7)
		Proximal joint of index finger	(B7,B8,C8,C7)
Middle finger distal joint	(R5,R5|6,R6,G6,G5)
		Middle finger middle joint	(G5,G6,B6,B5)
Proximal joint of middle finger	(B5,B6,C6,C5)
		Ring finger long joint	(R3,R3|4,R4,G4,G3)
Middle section of ring finger	(G3,G4,B4,B3)
		Proximal joint of ring finger	(B3,B4,C4,C3)
Little finger distal joint	(R1,R1|2,R2,G2,G1)
		Middle joint of little finger	(G1,G2,B2,B1)
Proximal joint of little finger	(B1,B2,C2,C1)
		Palm of hand	(C0,C1,...,C11)

It should be noted that, in the distal segment, 3 identification points are provided on the side of the finger tip, and the 3 identification points are adapted to the arc shape of the finger tip.

After the 3D drawing of Windows10 is used to complete the drawing of the detection points, pixel distortion occurs due to the influence of different video card resolutions during the image saving process, so that the distorted pixels of the image must be restored when the image is read. Specific values of pixel distortion reduction are shown in table 2.

TABLE 2

For a computer with a windows10 system as a current original system, after testing, the distorted red, green, blue and cyan pixels are all in the range shown in table 2. The restoration of the distorted data can be realized only by taking the distortion range as a criterion.

After the computer reads the image, the identification points are stored in the computer in the form of image coordinates of pixel points, each identification point comprises a plurality of pixel points, and the composition of each identification point pixel point is mainly related to the size of a drawing scale h selected when 3D drawing of Windows10 is carried out. Fig. 3 shows the shapes of the identification points corresponding to different drawing scales in the 3D drawing software of Windows10, where the shapes are composed of a plurality of pixel points and approximate to a circle. The drawing scale h is shown as the height of the marking point in the drawing, which also has a base width w, which is the number of pixels on the lowermost pixel side of the pattern. The highest drawing scale supported by Window 103D drawing is 100, and the specific drawing scale h and the base width w satisfy the corresponding relationship between table 3 and fig. 4. In the embodiment, the suitable drawing ruler h determined according to the size of the human hand is 15-20.

TABLE 3

After the image is scanned, because the identification points are numerous and each identification point comprises a plurality of pixel points, tens of millions of pixel coordinate data are stored in the computer, and under the condition of such a great amount of data, in the embodiment of the invention, the subsequent positioning and local area calculation taking the central pixel point of each identification point as the key position of the hand can be selected, so that more calculation amount can be reduced. Therefore, the key step in this embodiment is to find the center pixel point of each identification point.

The drawing scale h and the bottom edge width w are two parameters necessary for calculating the central pixel of the identification point, and the center of the detection point can be uniquely determined as long as the rightmost pixel point coordinate (x, y) of the bottom edge of the identification point is determined. We take the detection point with drawing scale 15 as an example, and the center of the diagram is shown in fig. 5. As can be seen from FIG. 5, when the bottom edge length of the detecting point is w, the image coordinates of the leftmost and rightmost pixels are (x-w-1, y) and (x, y), respectively, and the middle point coordinate of the bottom edge is (x- (w-1)/2, y). Because the drawing ruler h is equal to the axial length of the symmetry axis, the intersection point of the longitudinal and transverse symmetry axes is the mass center of the regular detection point. The centroid coordinate c of the identification point in the image coordinate system is (x- (w-1)/2, y + 1-h/2).

As an alternative embodiment, when the identification point is marked manually, it may occur that the identification point is not in a standard shape as shown in fig. 3, and the above method of calculating the center pixel can still be used to calculate the approximate center pixel coordinate, i.e. the centroid coordinate, so that a small amount of tolerable error occurring to the drawing personnel can be accommodated to improve the robustness of the model.

Based on the above calculation method, the key step is to determine the coordinates (x, y) of the rightmost end of the bottom edge of each identification point. The embodiment of the invention roughly divides the identification points with different position types through colors, distinguishes the identification points from the identification points one by one aiming at the identification points with a certain color, can determine the coordinates (x, y) at the rightmost end of the bottom edge of the distinguished identification points and further obtains the position of the central pixel point of the identification point.

Based on the above thought, in this embodiment, as shown in fig. 6, the obtaining of the coordinates (x, y) of the rightmost end of the bottom edge of each point location is performed as follows:

firstly, acquiring all point location coordinate sets of a certain color;

secondly, sorting all point positions in the set according to the size of a vertical coordinate;

thirdly, positioning w continuous coordinate points at the end part of the end part with the larger vertical coordinate of the queue according to the queuing sequence, wherein the point with the largest horizontal coordinate in the w coordinate points is the coordinate at the rightmost end of the bottom edge of the point position to which the w coordinate points belong;

fourthly, obtaining all coordinate points corresponding to the point positions to which the w coordinate points belong to form current point position coordinates, removing the current point position coordinates from the original point position coordinate set to form a new set, and replacing the original point position coordinate set;

and repeating the second step to the fourth step until the coordinates of the rightmost end of the bottom edges of all the point positions of the color are determined.

In fig. 6, a is a set of coordinates of all detected point pixels of a certain color; b is a set for recording the pixel coordinates of the detection point; and C is the set of the centroid coordinates of all the detection points of a certain color.

As an alternative embodiment, the sets C of center pixels corresponding to different colors may be obtained by the above method. The correct number of center pixels for red, green, blue, cyan are 15, 10, 8 and 12, respectively. And judging whether the detection points corresponding to the colors are more or less drawn or whether the detection points are seriously deformed beyond the allowable error range by judging the difference between the number of the centroids in the set of different colors C and the correct value.

More specifically, in the embodiment of the present invention, it is a difficult point to be solved by the present invention to further obtain pixels belonging to the same identification point under the condition of a plurality of pixel points. Since the embodiment of the present invention inevitably satisfies that a certain gap exists between the identification points in a real-time process, that is, the identification points and the identification points at least satisfy the interval of 1 pixel point, under such a condition, the embodiment of the present invention provides a method for clustering pixels, as shown in fig. 7 and 8, the method for obtaining all coordinate points corresponding to the point location to which the w coordinate points belong is as follows:

defining an entry point P0 and an upward movement determination point F _ U as parameters associated with a row, where the entry point P0 represents an initial determination point as to whether a coordinate point of the row in which the entry point P0 is located belongs to a corresponding point, and the upward movement determination point F _ U is used to determine whether the entry point P0 moves upward for updating; the initial entry point P0 is a starting point of the point to which the w coordinate points belong, and the initial upward movement judgment point F _ U is another point in the same row as the initial entry point P0;

when judging for each current line, the following processes are carried out:

judging whether the entry point P0 belongs to a target point, and respectively judging whether the adjacent coordinate points of the line where the entry point P0 belongs are the current color or not one by one leftwards and rightwards by taking the position of the entry point P0 as a starting point until the judgment of all the coordinate points of the current line is finished;

in the judging process, if a certain coordinate point of the current row is the current color, judging that the coordinate point belongs to the target point position, otherwise, finishing the direction judgment of the current row; when the coordinate point belongs to the target point position, further judging whether a coordinate point above the coordinate point is a current color or not, if so, taking the coordinate point above the coordinate point as an entry point P0 of a previous line of the current line and continuing to judge the coordinate points of the current line one by one, and if not, directly continuing to judge the coordinate points of the current line one by one;

when the left and right directions of the current row are judged to be finished, if the current entering point P0 is satisfied to belong to the current color at the moment and the current entering point P0 is different from the upward shifting judgment point F _ U, the judgment is finished, if the conditions are not satisfied, the upward shifting judgment point F _ U is updated according to the coordinate position of the current entering point P0, and the judgment of a new row is started by taking the current entering point P0 as the judgment starting point of the row.

In fig. 8, a set a is a label of a set of a certain color, and a set B represents a set of pixels in the same dot. The initial upward shift determination point F _ U is positioned to the right of the initial entry point P0. And F _ L and F _ R are defined as flag bits for the start and end of the left and right shift, respectively, where a flag bit of 0 indicates the start of the shift and a flag bit of 1 indicates the end of the direction.

According to the judging method, because a gap is inevitably formed between the actual point location and the point location, the step-by-step judgment is adopted, whether the line is finished or not can be quickly determined through convenient color judgment in the left branch and the right branch, whether pixels in the target point location exist in the previous line can be quickly determined through color judgment on the corresponding point of the previous line of the current judging point in the judging process of the current line, the entering point location P0 of the previous line is determined, and the entering point location P0 of the previous line is continuously updated along with the current judgment until the current line is judged to be finished, so that the latest entering point location P0 of the previous line is obtained. The position of the last row of entry point P0 is related to the number of pixels in the actual point, and the specific position does not affect the implementation of the above-mentioned clustering scheme of the present invention.

As an alternative embodiment, in the same drawing, the scales of all the identification points are the same, so that the pixel clustering method can be compatible with irregular identification points.

The pixel clustering algorithm finds out the pixel points belonging to the current clustering point by one step each time of displacement and combining with a color judgment mode to finish the clustering of the identification point, removes the determined identification point under the color, and repeats the process, namely clustering point by point, thereby finishing the clustering of all the identification points under the color.

After the positions of the central pixels of all the identification points are determined, the central pixels need to be further corresponding to the positions of the hands. Similarly, since the central pixel position exists in a coordinate form, the central pixel needs to be associated with the key position of the hand in combination with the physiological structure of the hand. Especially, in consideration of the serious inclination of the hand during photographing or scanning, such as the state shown in fig. 2, there is no obvious satisfactory association relationship between each central pixel point and the key structure of the hand, which results in difficult positioning. In the embodiment of the invention, on the basis of distinguishing the marking points with different position types through colors, the adjustment of the hand direction is further provided, and each marking point (represented by respective central pixels) is more conveniently corresponding to the key position of the hand. In the embodiment of the invention, the whole image is not rotated, but the coordinates of the central pixel are rotated, so that the same effect of all the whole image adjustments can be realized only by rotating 45 point coordinate points, and the workload is greatly reduced.

According to the embodiment of the invention, the hands are turned to be in the vertical state, so that the condition that the arrangement of the identification points is in one-to-one correspondence with key structures of the hands from left to right along with the increase of the abscissa of the identification points under the same color is necessarily met. Specifically, in the embodiment of the present invention, the center of each point position is turned into the hand-vertical state around the rotation point, and the process is performed as follows:

the method comprises the steps of firstly, obtaining an image coordinate of the rightmost end of the bottom edge of each point location, the length w of the bottom edge of the point location and a scale h, calculating the image coordinate of the center of the point location, and converting the image coordinate of the center of the point location into Cartesian coordinates (x, y);

secondly, acquiring 4 point locations at two joints corresponding to a middle section forming the index finger, connecting every two point locations corresponding to each joint to obtain a connecting line of midpoints of the two connecting lines, and acquiring an included angle alpha between the connecting line and a vertical line, wherein the included angle alpha is shown in figure 2 specifically;

thirdly, the coordinates (X, Y) of the center of the rotated point location are calculated according to the following formula

X＝(x-xx)*cos(a)-(y-y₀)*sin(a)+x₀，

Y＝(x-x₀)*sin(n)-(y-y₀)*cos(a)+y₀，

Wherein (x)₀,y₀) Is the cartesian coordinates of the rotation point.

Before rotation, the specific method for converting the image coordinate points into Cartesian coordinates is as follows: the cartesian abscissa is equal to the abscissa of the image coordinate point and the cartesian ordinate is equal to the ordinate of the overall height of the photograph or scan minus the image coordinates.

In the embodiment of the present invention, the rotation point is the centroid of the palm, and therefore, in the embodiment of the present invention, the cartesian centroid coordinates of the 12-cyan-point great wall polygon are calculated and taken as the center point (x) of the rotation₀,y₀)。

x_i,y_iRespectively the horizontal and vertical coordinate values of the centroid of the cyan point.

After the centroid is rotated, the detection points corresponding to the red, the green and the blue are arranged in a positive sequence according to the value of the abscissa of the coordinate, and then the centroid coordinate of each color related to each joint part can be accurately determined. For cyan dots, positive sequence arrangement is required to be carried out according to the size of a vertical coordinate, the centroid points of the first 10 detection points are taken, then the 10 centroid points are subjected to positive sequence arrangement according to the size of a horizontal coordinate, and the positive sequence centroid points at the moment correspond to C1-C10 one by one. However, in practice, it has been found that even if the gesture is vertically upward, the thumb may be horizontal, which results in C9 and C10, G9 and G10, and R9 and R10 not being able to judge the left-right relationship to each other according to the abscissa size. Although it can be accurately determined that the combination of the distal and proximal joints of the thumb is composed of (R9, R9|10, R10, G10, G9), (G9, G10, C10, C9), respectively, it is not determined at which position of the distal or proximal joint of the thumb each point inside the combination point is specifically located.

In order to solve the problem that the real positions of the points in the combined point cannot be determined through the vertical and horizontal coordinates, an embodiment of the present invention provides a reverse ordering algorithm in the combined point as shown in fig. 9, that is, the method for completing the positioning of the corresponding position of the hand structure according to the type of the point location and the coordinates of the point location center is as follows:

firstly, acquiring all point location centers corresponding to a certain hand structure, and acquiring the centroid positions (x) of all the point location centers₀,y₀)；

Secondly, calculating the angle position beta of each point position center (x, y) relative to the centroid according to the following formula:

and thirdly, determining the relative position relation between the centers of the point positions according to the size of the angle position beta, and determining the corresponding relation between the hand structure and each point position according to the relative position relation to complete positioning.

After the positioning is completed, the area of the local position of the hand can be further calculated by obtaining a convex polygon composed of points belonging to a certain hand structure, and calculating the area of the certain hand structure according to the following formula:

wherein P is a pixel of the convex polygon,since this parameter is related to the pixel size of the hand image itself, the area of the local region is calculated by obtaining the product of the pixel size of the region and the area-pixel ratio of the hand image.

Specifically, the finger joints and palms of the single hand are shown in table 4, and after the reverse sorting algorithm in the combination point, the requirement for calculating the area of the convex polygon is already met. Assuming that after a certain combined point is subjected to reverse sorting, the coordinates of each point are respectively as follows:

((a₁，b₁)，(a₂，b₂)，...，(a_k，b_k)，...，(a_n-1，b_n-1)，(a_n，b_n))，，

it is obvious that a certain joint can be represented by a convex polygon formed by combining the n points, and the convex polygon can be solved into n-2A triangle. And calculating the area sum of n-2 triangles to obtain the area of the joint. We get the following (a)₁，b₁) Is a common point of triangle, then is composed of (a)₁，b₁)，(a_k，b_k)，(a_k+1，b_k+1) The number of pixels covered by the three points is as follows:

the number of pixels contained in the convex polygon is:

particularly, when the certain hand structure is at the distal joint position, the hand structure has an arc structure, the distal knuckle joint is subjected to pentagon fitting, the area value of the hand structure is slightly smaller than the real value, so that the convex algorithm is adopted to calculate the area, a small amount of area loss exists, the area loss is compensated by a coefficient when the area of the hand structure is calculated, therefore, the distal knuckle joints of testers with different sexes and ages are tested by adopting a hierarchical sampling method, and the result shows that the error between 1.1 time of the area of the pentagon fitting and the real value is within +/-5%, so that when the area of the hand structure is calculated, the hand structure is provided with an arc structure, the area value of the distal knuckle joint is slightly smaller than the real value, and the error between the area of the hand structure and the real value is within +/-5%

TABLE 4

In this embodiment, an electronic device is provided, comprising a memory in which a computer program is stored and a processor configured to run the computer program to perform the method in the above embodiments.

These computer programs may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks, and corresponding steps may be implemented by different modules.

The programs described above may be run on a processor or may also be stored in memory (or referred to as computer-readable media), which includes both non-transitory and non-transitory, removable and non-removable media, that implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

The embodiment of the present invention is developed based on the software and hardware as follows:

a vertical transparent rectangular glass plate is arranged in front of the camera, and the glass plate is vertically placed on a movable support with the height capable of being freely adjusted. Black frames with the width and the height of 450mm and 300mm are horizontally pasted on the glass plate. Meanwhile, a vertical strip-shaped symmetrical shaft is arranged, the width and the height of the symmetrical shaft are respectively 10mm and 300mm, and the black frame is divided into two symmetrical frames. Each frame is provided with a vertically upward black dotted line gesture placing area. Before shooting, the camera needs to be adjusted to enable the shooting area to be just matched with the black frame, and the shooting focus is located at the centroid position of the black frame.

After the preparation work before shooing was accomplished, the region was put at the black dotted line gesture of glass board to the tester left and right hands pressure respectively, and the hand compresses tightly the glass board, and the hand joint can not the perk, will ensure that the hand striation can be discerned by the manual work.

After shooting, the pictures are automatically sent to a cloud processor or a local processor, the central axis of the black frame is automatically identified by an algorithm, and the black frame is cut into two pieces symmetrically. Drawing of the left-hand and right-hand key points is completed manually by using Windows103D drawing software. The drawing scale can be 1-60 in principle, but according to practical experience, the scale is more preferably 15 or 20 for key point drawing.

After drawing is finished, the picture is automatically input into the server, the area calculation is automatically finished by the algorithm, and the result is automatically stored in an Oracle database according to the picture name. And for unqualified pictures, the algorithm automatically screens and feeds back the result to a responsibility plotter. In specific implementation, the algorithm is built through an Ali cloud platform development environment and is clouded. The algorithm of the embodiment of the invention supports high concurrency, is influenced by the performance of a cloud server or a local server, and can realize the high concurrency operation with the process number of 30 at most at present, namely the calculation of finger joints and palm areas of 30 hands is processed at the same time.

In order to compare the feasibility of the method and the performance of the system, the same tester is invited, and the calculation of the area of the knuckles and the palm of the two hands of the tester is completed by using the method in the embodiment of the invention. Then, a completely artificial mode is used for specifically drawing the hand contour, and the number of grids in the knuckle is searched to determine the area. In addition, in order to ensure the objectivity of performance comparison, multiple experts are invited to perform high-precision manual drawing and integral calculation on hands in a laboratory, and the mean value of the calculation results of the multiple experts is regarded as a theoretical value. Table 5 gives the results of the laboratory calculations, the complete manual and the algorithm of this patent, respectively.

TABLE 5

As can be seen from table 5 above, the error between the calculated result and the laboratory calculated result in the embodiment of the present invention is within ± 3%, while the error between the manual drawing of the contour number grid is even more than 8%, such as 8.78%, 8.87%, 10.67% for the errors between the manual and laboratory calculated results of the left index finger proximal node, the right thumb proximal node, and the right little finger proximal node in table 4. Because the hand contour drawing by manual drawing has larger error, the algorithm of the embodiment of the invention has more accurate calculation than manual calculation. Secondly, manual table look-up needs high concentration and is easy to fatigue, the accuracy of the check is greatly reduced along with the increase of workload, and the test shows that 3 minutes are consumed for storing area results from the acquisition of images of two hands in a database, and 45 minutes are consumed for manual table look-up, so that the difference between the calculation efficiency of manual work and the calculation efficiency of the embodiment of the invention is large.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.