阅读说明：本技术 一种基于单目相机的头部三维坐标的摔倒检测方法 (Method for detecting tumbling of head three-dimensional coordinate based on monocular camera ) 是由 祝敏航 徐晓刚 曹卫强 王军 李玲 刘雪莹 于 2021-08-27 设计创作，主要内容包括：本发明公开了一种基于单目相机的头部三维坐标的摔倒检测方法,首先估计单目视频中头部三维坐标,再根据头部三维坐标的变化检测人员是否摔倒。单目头部三维坐标估计通过相机标定获得相机内参矩阵,通过头部检测获得头部在图像中的矩形框,利用相机内参、头部矩形框和预设的头部基本几何参数构建头部三维坐标求解方程,求解在相机坐标系下的头部三维坐标。通过系统标定获得世界坐标系到相机坐标系的变换矩阵,经过坐标变换,得到在世界坐标系下的头部三维坐标。当头部坐标z分量小于一定阈值时判定人员为摔倒状态。本方法利用单个摄像头即可估计头部的三维坐标,并判断人员是否摔倒,硬件成本低,可广泛应用于智能监控等领域。(The invention discloses a monocular camera-based method for detecting falling of a three-dimensional head coordinate. Monocular head three-dimensional coordinate estimation obtains a camera internal reference matrix through camera calibration, obtains a rectangular frame of the head in an image through head detection, constructs a head three-dimensional coordinate solving equation by utilizing the camera internal reference, the head rectangular frame and preset head basic geometric parameters, and solves the head three-dimensional coordinate under a camera coordinate system. And obtaining a transformation matrix from the world coordinate system to the camera coordinate system through system calibration, and obtaining the head three-dimensional coordinates in the world coordinate system through coordinate transformation. And when the z component of the head coordinate is smaller than a certain threshold value, judging that the person is in a falling state. The method can estimate the three-dimensional coordinates of the head by using a single camera, judges whether personnel fall down, has low hardware cost, and can be widely applied to the fields of intelligent monitoring and the like.)

1. A method for detecting falling of a head based on a monocular camera three-dimensional coordinate is characterized by comprising the following steps:

s1, calculating the three-dimensional coordinates of the head in the world coordinate system in the monocular image, and the method comprises the following steps:

s11, calibrating the camera by using the chessboard board to obtain the internal reference matrixM；

S12, carrying out system calibration to obtain a transformation matrix from the world coordinate system to the camera coordinate system ^w T _c ；

S13, approximating the head to a cuboid, and giving basic geometrical parameters of the head, including the length, the width, the height and the orientation of the head;

s14, detecting the head by using a target detection algorithm to obtain a rectangular frame representing the position of the head;

s15, constructing a linear equation for solving the three-dimensional coordinates of the head by using the internal reference matrix, the rectangular frame and the basic geometric parameters, and comprising the following steps:

s151, establishing a head three-dimensional coordinate system with homogeneous coordinates ofQDefining the homogeneous coordinates of the head image coordinate system asq；

S152, according to projection transformation, a three-dimensional coordinate solving equation of the head under a camera coordinate system is constructed, wherein the projection transformation equation is as follows:

q=MWQ (3)

whereinMIs an internal reference matrix, and is a reference matrix,W=[Rt]setting the coordinate axis direction of the head three-dimensional coordinate system to be consistent with the world coordinate system for the transformation matrix from the camera coordinate system to the head three-dimensional coordinate system, and rotating the matrixRGiven by the system calibration result in S12, a translation vectortObtaining the three-dimensional coordinates of the head under a camera coordinate system;

s153, arranging the form of a projection transformation formula:

(4)

wherein；

S154, establishingqAndthe corresponding point pair of (c) is defined as follows (x)_min, )、(x_max, )、(y_min, )、(y_max, ) Wherein x is_min、x_max、y_min、y_maxGiven by the rectangular frame of the head position obtained in said S14, 4 point coordinates of the rectangular frame are respectively represented,、、、given by the dimensional parameters of the cuboid given by S13, respectively、、、Wherein L, W, H denotes the length, width, height of the head, respectively;

s155, simplifying the corresponding points in the equation (4) to obtain a linear equation:

At=b (5)

wherein A is a data matrix and b is an observation vector;

s16, obtaining the head three-dimensional coordinates under the camera coordinate system through a linear equation;

s17, multiplying the head three-dimensional coordinate and the transformation matrix by a matrix to obtain the head three-dimensional coordinate under a world coordinate system;

s2, calculating the acceleration of the head based on the monocular image continuous frame difference, and calculating the component in the three-dimensional coordinates of the head in the world coordinate systemzAnd when the acceleration of the head is greater than the over-acceleration threshold value within a short preset time, the person is judged to be in a falling state, and the component z represents the distance from the head to the ground.

2. The monocular camera-based fall detection method for head three-dimensional coordinates according to claim 1, wherein the internal reference matrix is a matrix of three-dimensional coordinates，f _x 、f _y 、c _x 、c _y Respectively the focal length of the cameraxComponent, camera focal lengthyComponent, camera principal pointxComponent, camera principal pointyAnd (4) components.

3. The monocular camera-based fall detection method for the three-dimensional coordinates of the head according to claim 2, wherein the simplified linear equation in S155 is expanded as follows:

wherein the content of the first and second substances,，。

4. the method of claim 1, wherein the linear equation is solved in step S16 to obtain the three-dimensional coordinates of the head in the camera coordinate systemt=(A ^T A)^-1 A ^T b。

5. The method of claim 1, wherein the three-dimensional head coordinate system in the world coordinate system is obtained by coordinate transformation in S17T _w = ^w T _c t。

6. The method of claim 1, wherein the step S151 is to establish a three-dimensional head coordinate system with XYZ axes respectively parallel to XYZ axes of a world coordinate system, the origin of the XYZ axes being at the center of the head, and homogeneous coordinates of a point in the three-dimensional head coordinate system are:

Q=[XYZ 1] ^T (1)。

7. the method of claim 1, wherein the head in the S151 has homogeneous coordinates in an image coordinate system corresponding to:

q=[x,y,w] ^T (2)

wherein the content of the first and second substances,representing the x-component of the image coordinates,representing the image coordinate y component and w the scale.

8. The monocular camera-based fall detection method for the three-dimensional coordinates of the head according to claim 1, wherein the S12 comprises the steps of:

s121, establishing a world coordinate system on the ground, taking one point on the ground as an origin, enabling X, Y, Z axes of the world coordinate system to pass through the origin and be vertical to each other, enabling an X axis and a Y axis to be located on the ground, and enabling a Z axis to be vertical to the ground;

s122, arranging a group of non-collinear points on the ground, and recording three-dimensional coordinates of the points in a world coordinate system and two-dimensional coordinates of the points in an image coordinate system;

s123, solving a transformation matrix from the world coordinate system to the camera coordinate system by utilizing a PnP algorithm ^w T _c 。

9. The method of claim 1, wherein the length, width and height of the head in S13 are preset directly.

Technical Field

The invention relates to the field of computer vision, in particular to a method for detecting falling of a head in three-dimensional coordinates based on a monocular camera.

Background

In the field of intelligent monitoring, the timely detection of abnormal behaviors (such as falls) of personnel is a basic monitoring requirement. As a data source for detecting abnormal behaviors, the three-dimensional coordinate information of the head can effectively assist a detection algorithm to improve the detection rate of the abnormal behaviors. In indoor monitoring applications, too low head height and too high head acceleration indicate a greater probability of a fall.

Common methods for acquiring three-dimensional coordinates of the head include a depth camera, a binocular camera, and the like. However, in practical security applications, it is difficult to deploy depth cameras and binocular cameras due to cost issues.

Disclosure of Invention

In order to solve the defects of the prior art, realize the purpose of ensuring the detection accuracy while saving the cost without changing the prior monitoring scheme of the prior monocular camera, the invention adopts the following technical scheme:

a method for detecting falling of a head based on three-dimensional coordinates of a monocular camera comprises the following steps:

s1, calculating the three-dimensional coordinates of the head in the world coordinate system in the monocular image, and the method comprises the following steps:

s11, calibrating the camera by using the chessboard board to obtain the internal reference matrixM；

S12, carrying out system calibration to obtain a transformation matrix from the world coordinate system to the camera coordinate system ^w T _c ；

S13, approximating the head to a cuboid, and giving basic geometrical parameters of the head, including the length, the width, the height and the orientation of the head;

s14, detecting the head by using a target detection algorithm to obtain a rectangular frame representing the position of the head;

s15, constructing a linear equation for solving the three-dimensional coordinates of the head by using the internal reference matrix, the rectangular frame and the basic geometric parameters, and comprising the following steps:

s151, establishing a head three-dimensional coordinate system with homogeneous coordinates ofQDefining the homogeneous coordinates of the head image coordinate system asq；

S152, according to projection transformation, a three-dimensional coordinate solving equation of the head under a camera coordinate system is constructed, wherein the projection transformation equation is as follows:

q=MWQ (3)

whereinMIs an internal reference matrix, and is a reference matrix,W=[Rt]for the transformation matrix from the camera coordinate system to the head three-dimensional coordinate system, considering that the head rotation does not affect the three-dimensional coordinates of the head, the orientation of the head is directly set to be along the world coordinate system, namely, the coordinate axis direction of the head three-dimensional coordinate system is consistent with the world coordinate system, so that the rotation matrixRThe translation vector is directly given by the system calibration result in the S12tObtaining the three-dimensional coordinates of the head under a camera coordinate system;

s153, arranging the form of a projection transformation formula:

(4)

wherein；

S154, establishingqAndsince the monitoring camera is generally located obliquely above the head,qandthe corresponding point pairs of (c) are defined as follows (x)_min,)、(x_max,)、(y_min,)、(y_max,) Wherein x is_min、x_max、y_min、y_maxGiven by the rectangular frame of the head position obtained in said S14, 4 point coordinates of the rectangular frame are respectively represented,、、、given by the dimensional parameters of the cuboid given by S13, respectively、、、Wherein L, W, H denotes the length, width, height of the head, respectively;

s155, simplifying the corresponding points in the equation (4) to obtain a linear equation:

At=b (5)

wherein A is a data matrix and b is an observation vector;

s16, obtaining the head three-dimensional coordinates under the camera coordinate system through a linear equation;

s17, multiplying the head three-dimensional coordinate and the transformation matrix by a matrix to obtain the head three-dimensional coordinate under a world coordinate system;

s2, calculating the acceleration of the head based on the monocular image continuous frame difference, and calculating the component in the three-dimensional coordinates of the head in the world coordinate systemzAnd when the acceleration of the head is greater than the over-acceleration threshold value within a short preset time, the person is judged to be in a falling state, and the component z represents the distance from the head to the ground.

Further, the internal reference matrix，f _x 、f _y 、c _x 、c _y Respectively the focal length of the cameraxComponent, camera focal lengthyComponent, camera principal pointxComponent, camera principal pointyAnd (4) components.

Further, the simplified linear equation in S155 is expanded as follows:

wherein the content of the first and second substances,，。

further, in S16, the linear equation is solved to obtain the three-dimensional coordinates of the head in the camera coordinate systemt=(A ^T A)^-1 A ^T b。

Further, in S17, the three-dimensional coordinates of the head in the world coordinate system are obtained through coordinate transformationT _w = ^w T _c t。

Further, in the S151, a head three-dimensional coordinate system is established, XYZ axes of which are respectively parallel to XYZ axes of a world coordinate system, an origin of which is at a head center, and homogeneous coordinates of a point in the head three-dimensional coordinate system are:

Q=[XYZ 1] ^T (1)

further, the homogeneous coordinates of the head in S151 in the image coordinate system are:

q=[x,y,w] ^T (2)

wherein the content of the first and second substances,representing the x-component of the image coordinates,representing the image coordinate y component and w the scale.

Further, the S12 includes the following steps:

s121, establishing a world coordinate system on the ground, taking one point on the ground as an origin, enabling X, Y, Z axes of the world coordinate system to pass through the origin and be vertical to each other, enabling an X axis and a Y axis to be located on the ground, and enabling a Z axis to be vertical to the ground;

s122, arranging a group of non-collinear points (9 points) on the ground, and recording three-dimensional coordinates of the points in a world coordinate system and two-dimensional coordinates of the points in an image coordinate system;

s123, solving a transformation matrix from the world coordinate system to the camera coordinate system by utilizing a PnP algorithm ^w T _c 。

Further, the length, width and height of the head in S13 are directly preset and given considering the close geometric sizes of the heads of different people.

The invention has the advantages and beneficial effects that:

according to the invention, under the condition that the existing monitoring hardware equipment is not changed, the three-dimensional coordinates of the head are estimated through the monocular camera, and whether the head falls down or not is judged through the three-dimensional coordinates of the head and the acceleration of the head, so that the application cost is saved, and meanwhile, the detection precision is ensured.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

FIG. 2 is a schematic diagram of the system calibration of the present invention.

Fig. 3 is a diagram illustrating the corresponding point pairs in the present invention.

Detailed Description

The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.

As shown in fig. 1, a method for detecting a fall based on three-dimensional coordinates of a head of a monocular camera includes the following steps:

the method comprises the following steps: in the monocular image, calculating three-dimensional coordinates of the head under a world coordinate system, and the method comprises the following steps:

step 1.1: calibrating the monitoring camera by using the chessboard board to obtain an internal reference matrix，f _x 、f _y 、c _x 、c _y Respectively the focal length of the cameraxComponent, camera focal lengthyComponent, camera principal pointxComponent, camera principal pointyComponent, e.g.。

Step 1.2: as shown in fig. 2, the camera is fixed on the wall, the mark points are arranged on the ground at an interval of 2m in a 3 × 3 array, one of the array points is taken as an origin, two vertical directions of the array are taken as XY axes, a world coordinate system is established with a Z axis vertical to the ground, the camera takes an image containing the array,method for solving transformation matrix from world coordinate system to camera coordinate system by utilizing PnP algorithm ^w T _c E.g. of。

Step 1.3: as shown in fig. 3, given that the head length, width and height LWH are all 0.2m, the coordinate axis direction of the given head coordinate system coincides with the world coordinate system,、、、are respectively [ -0.1-0.10.1] ^T 、[0.1 0.1 0.1] ^T 、[-0.1 0.1 0.1] ^T 、[-0.1 -0.1 -0.1] ^T 。

Step 1.4: using a target detection algorithm YOLO to carry out head detection to obtain a head rectangular frame [ x ]_min, x_max, y_min, y_max]E.g. [150, 165, 300, 315 ]]。

Step 1.5: the method comprises the following steps of constructing a linear equation for solving the three-dimensional coordinates of the head by using an internal reference matrix, a rectangular frame and basic geometric parameters, and comprising the following steps of:

step 1.5.1: establishing a head three-dimensional coordinate system, wherein the homogeneous coordinate is as follows:

Q=[XYZ 1] ^T (1)

the homogeneous coordinates defining the head image coordinate system are:

q=[x,y,w] ^T (2)

wherein the content of the first and second substances,representing the x-component of the image coordinates,representing the image coordinate y component and w the scale.

Step 1.5.2: according to projection transformation, a three-dimensional coordinate solving equation of the head under a camera coordinate system is constructed, wherein the projection transformation equation is as follows:

q=MWQ (3)

whereinMIs an internal reference matrix, and is a reference matrix,W=[Rt]for the transformation matrix from the camera coordinate system to the head three-dimensional coordinate system, considering that the head rotation does not affect the three-dimensional coordinates of the head, the orientation of the head is directly set to be along the world coordinate system, namely, the coordinate axis direction of the head three-dimensional coordinate system is consistent with the world coordinate system, so that the rotation matrixRThe translation vector is directly given by the system calibration result in the step 1.2tIs the three-dimensional coordinates of the head under the camera coordinate system.

Step 1.5.3: and (3) finishing the form of the projective transformation formula:

(4)

wherein。

Step 1.5.4: establishingqAndsince the monitoring camera is generally located obliquely above the head, as shown in fig. 3, the corresponding point pair of (b) will beqAndto the coordinates (x) of_min,)、(x_max,)、(y_min,)、(y_max,) The belt-in type (4),、、、given by the dimensional parameters of the cuboids given in step 1.3, respectively、、、Wherein L, W, H denotes the length, width, height of the head, respectively, e.g. obtaining。

Step 1.5.5: in the equation (4), the corresponding points are simplified to obtain a linear equation:

At=b (5)

wherein A is a data matrix, b is an observation vector, and the extension form of the simplified linear equation is as follows:

wherein the content of the first and second substances,，。

step 1.6: solving the above formula to obtain the three-dimensional coordinates of the head under the camera coordinate system, and obtaining the three-dimensional coordinates of the head under the camera coordinate system by solving the linear equationt=(A ^T A)^-1 A ^T bE.g., [ -0.88-0.331.83] ^T 。

Step 1.7: the system calibration result is utilized to carry out coordinate transformation to obtain the three-dimensional coordinates of the head under the world coordinate systemT _w = ^w T _c tE.g. [ 1.420.981.44] ^T 。

Step two: calculating the acceleration of the head by continuous frame difference, and when the z component of the three-dimensional coordinate of the head in the step 1.7 is less than the threshold value of 0.2m and the acceleration of the head is more than 2m/s in nearly 1 second²If not, the state is a non-tumbling state.

The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.