阅读说明：本技术 一种利用线性回归模型的人脸视频心率检测方法 (Human face video heart rate detection method using linear regression model ) 是由 谢巍 吴少文 魏金湖 周延 陈定权 许练濠 卢永辉 于 2020-12-09 设计创作，主要内容包括：本发明公开了一种利用线性回归模型的人脸视频心率检测方法,该方法包括以下步骤：S1、从可见光人脸视频图像的三基色通道观测信号分离出三个独立信号源；S2、将分离出的三个独立源信号通过FFT转为频域信号,选择其中更具有最大功率谱幅值的源信号作为脉搏源信号,计算出心率值；S3、创建心率数据集,利用线性回归模型修正从人脸视频信号提取出的心率值。本发明针对人脸图像心率检测方法易受到人脸晃动,光线变化等外界因素影响的问题,提出通过一种创建心率数据集,利用线性回归模型,修正通过人脸视频获取心率值的方法,提高利用人脸视频检测心率方法的准确率。(The invention discloses a human face video heart rate detection method by utilizing a linear regression model, which comprises the following steps: s1, separating three independent signal sources from three primary color channel observation signals of the visible light human face video image; s2, converting the three separated independent source signals into frequency domain signals through FFT, selecting the source signal with the maximum power spectrum amplitude as a pulse source signal, and calculating a heart rate value; and S3, creating a heart rate data set, and correcting the heart rate value extracted from the human face video signal by using a linear regression model. The invention provides a method for correcting a heart rate value obtained through a face video by establishing a heart rate data set and utilizing a linear regression model, aiming at the problem that a face image heart rate detection method is easily influenced by external factors such as face shaking, light change and the like, and the accuracy of the method for detecting the heart rate by utilizing the face video is improved.)

1. A human face video heart rate detection method using a linear regression model is characterized by comprising the following steps:

s1, acquiring a human face region of interest from a visible light human face video image, and separating three independent signal sources based on a tricolor channel observation signal;

s2, converting the three separated independent source signals into frequency domain signals through FFT, selecting the source signal with the maximum power spectrum amplitude as a pulse source signal, and calculating a heart rate value;

and S3, creating a heart rate data set, and correcting the heart rate value extracted from the human face video signal by using a linear regression model.

2. The method of detecting human face video heart rate using linear regression model as claimed in claim 1,

the step of obtaining the region of interest of the face is to obtain a video of a face image by using a visible light camera, detect the positions of key points of the face in the video image of the face by adopting a Multi-task convolution (MTCNN) and determine the region of interest of the face (ROI).

3. The method as claimed in claim 2, wherein the R, G, B three primary color channel of the ROI area in each frame is separated, the average value of all three components of the pixels in the ROI area is taken, and the three average values in each frame form three time series observation signals x₀(t)、x₁(t)、x₂(t) using the FastICA algorithm, three independent source signals are separated from the three observed signals.

4. The method for detecting the heart rate of the human face video by using the linear regression model as claimed in claim 3, wherein the FastICA algorithm is as follows:

suppose there are three underlying source signals s₀(t)、s₁(t)、s₂(t) the observation signal is X (t) ═ x₀(t)，x₁(t)，x₂(t)]^TThe base source signal is S (t) ═ s₀(t)，s₁(t)，s₂(t)]^TThe observed signal X (t) being a linear combination of the basic source signals S (t), i.e.

X(t)＝A·S(t)

Where A is the mixing matrix, the mixing matrix is made approximately equal to A by finding the separation matrix W^-1So that the source signal approximation signal y (t) ═ W × x (t) approximates the source signal, the FastICA algorithm maximizes the objective function by mixing the negative entropy of the signal into the objective function, which is as follows:

J(W)＝[E{G(W^TZ)}-E{G(V)}]²

wherein the nonlinear function G (V) is V³Nonlinear function G (W)^TZ)＝(W^TZ)³(ii) a Z ═ vx (t), V is the whitening matrix, W is the separation matrix, E { G (W)^TZ) -E { G (V) } is a mathematical expectation.

5. The method for detecting human face video heart rate by using linear regression model as claimed in claim 4, wherein in step S2, the separated independent source signal is transformed to frequency domain by FFT for analysis, the FFT transformation formula is as follows:

f (t) represents an independent source signal in the time domain, j represents an imaginary unit, t represents time, and w represents an angular frequency.

6. The method as claimed in claim 5, wherein the current heart rate value is calculated according to a frequency value corresponding to a maximum peak value of a pulse source signal spectrum, and the heart rate calculation formula is as follows:

HR＝F_max*60

wherein, F_maxIs the frequency corresponding to the maximum peak.

7. The method as claimed in claim 6, wherein the heart rate data set in step S3 is obtained by capturing a plurality of human face video clips of several subjects by using a visible light camera, and capturing a heart rate signal by using a physiological sensor while capturing the video clips, the heart rate signal captured by the physiological sensor being used as a dependent variable y_iThe heart rate signal obtained based on the face video is used as an explanatory variable x_iThe collected data set isWherein n is the number of data sets.

8. The method for detecting the heart rate of the human face video by using the linear regression model as claimed in claim 7, wherein the linear regression model is in the form of:

y_i＝β₀+β₁x_i+ε_i

wherein i is 1, …, n, beta₀Representing the intercept, beta, of a linear regression model₁Denotes the slope, ε_iIndicating an error;

assuming an expected value of error of 0 for all data sets i, the common least squares method is used to estimate β₀And beta₁The residual sum of squares is minimized as follows:

β₀and beta₁The calculation formula of (a) is as follows:

wherein x represents a heart rate signal data set acquired by a physiological sensor, y represents a heart rate signal data set acquired by a human face video, Cov (x, y) represents covariance of x and y, Var (x) represents variance of x, and x represents variance of x_iRepresenting the heart rate signal, y, acquired by the ith physiological sensor_iRepresenting the heart rate signal obtained for the ith personal video.

9. The method for detecting the heart rate of the human face video by using the linear regression model as claimed in claim 8, wherein the heart rate value extracted in step S3 is specifically:

randomly selecting partial data in the data set, calculating a heart rate value by using a human face video estimation method, and calculating beta according to the step S302₀And beta₁The remaining data is used as a prediction estimate, and the fitting effect is evaluated by using the goodness-of-fit index.

10. The method as claimed in claim 9, wherein the goodness-of-fit index R is a measure of the heart rate of the human face video using a linear regression model²The formula is as follows:

where Var (y) represents the variance of y.

Technical Field

The invention relates to the field of image processing and heart rate detection, in particular to a human face video heart rate detection method by using a linear regression model.

Background

With the improvement of life of people, the health is a major concern of people. The heart rate value is too high or too low, which indicates that the human body is in a sub-health state or even a dangerous state, so the heart rate is one of the most important vital signs of the human body, and the detection of the heart rate is also paid more attention. In recent years, contact type heart rate detection devices have been developed rapidly, and although many improvements are made in the direction of small size, convenient measurement and the like, they all need to make direct physical contact with a measurer, and the contact type sensor recording manner measurement process is cumbersome and may cause discomfort to patients, especially to special people such as babies who are born. Therefore, the non-contact heart rate detection method utilizing the iPGP principle has wider application prospect in the medical field, the family health prevention and the like. Some existing researches (Huang-Shen-Feng-Zhou-nan-Guo-Hou-Sichuan Liu-hong Wang, a non-contact heart rate detection method based on camera shooting and a device thereof, China, CN201810236275.3 and 2018) also utilize a non-contact heart rate detection method, but the result is inaccurate because the measurement result is easily influenced by factors such as the field, light change and shaking. In order to overcome the defects, the invention provides a linear regression model scheme, which can improve the accuracy of the heart rate value detected by the iPG principle, reduce the influence of external noise and ensure the accuracy of the measurement result.

Disclosure of Invention

In order to solve the problem that non-contact heart rate detection is easily influenced by external illumination, shaking and influenced results in the measuring process, the invention provides a human face video heart rate detection method using a linear regression model.

The invention is realized by at least one of the following technical schemes.

A human face video heart rate detection method using a linear regression model comprises the following steps:

s1, acquiring a human face region of interest from a visible light human face video image, and separating three independent signal sources based on a tricolor channel observation signal;

s2, converting the three separated independent source signals into frequency domain signals through FFT, selecting the source signal with the maximum power spectrum amplitude as a pulse source signal, and calculating a heart rate value;

and S3, creating a heart rate data set, and correcting the heart rate value extracted from the human face video signal by using a linear regression model.

Preferably, the step of obtaining the region of interest of the face is to obtain a video of the face image by using a visible light camera, detect the position of a key point of the face in the video image of the face by using a Multi-task convolution (MTCNN) to determine the region of interest of the face (ROI).

Preferably, the R, G, B three-primary color channel of the ROI area in each frame is separated, the average value of all three components of the pixels in the ROI area is taken, and the three average values in each frame form three time-series observation signals x₀(t)、x₁(t)、x₂(t) using the FastICA algorithm, three independent source signals are separated from the three observed signals.

Preferably, the FastICA algorithm is specifically as follows:

suppose there are three underlying source signals s₀(t)、s₁(t)、s₂(t) the observation signal is X (t) ═ x₀(t)，x₁(t)，x₂(t)]^TThe base source signal is S (t) ═ s₀(t)，s₁(t)，s₂(t)]^TThe observed signal X (t) being a linear combination of the basic source signals S (t), i.e.

X(t)＝A·S(t)

Where A is the mixing matrix, the mixing matrix is made approximately equal to A by finding the separation matrix W^-1So that the source signal approximation signal y (t) ═ W × x (t) approximates the source signal, the FastICA algorithm maximizes the objective function by mixing the negative entropy of the signal into the objective function, which is as follows:

J(W)＝[E{G(W^TZ)}-E{G(V)}]²

wherein the nonlinear function G (V) is V³Nonlinear function G (W)^TZ)＝(W^TZ)³(ii) a Z ═ vx (t), V is the whitening matrix, W is the separation matrix, E { G (W)^TZ) -E { G (V) } is a mathematical expectation.

Preferably, in step S2, the separated independent source signal is transformed to the frequency domain by FFT for analysis, where the FFT transformation formula is as follows:

f (t) represents an independent source signal in the time domain, j represents an imaginary unit, t represents time, and w represents an angular frequency.

Preferably, the current heart rate value is calculated according to a frequency value corresponding to the maximum peak value of the pulse source signal frequency spectrum, and the heart rate calculation formula is as follows:

HR＝F_max*60

wherein, F_maxIs the frequency corresponding to the maximum peak.

Preferably, the heart rate data set in step S3 is obtained by capturing a plurality of human face video clips of several subjects by using a visible light camera, and capturing a heart rate signal by using a physiological sensor while acquiring the video clips, wherein the heart rate signal captured by the physiological sensor is used as the dependent variable y_iThe heart rate signal obtained based on the face video is used as an explanatory variable x_iThe collected data set isWherein n is the number of data sets.

Preferably, the linear regression model has the following model form:

y_i＝β₀+β₁x_i+ε_i

wherein i is 1, …, n, beta₀Representing the intercept, beta, of a linear regression model₁Denotes the slope, ε_iIndicating an error;

assuming an expected value of error of 0 for all data sets i, the common least squares method is used to estimate β₀And beta₁The residual sum of squares is minimized as follows:

β₀and beta₁The calculation formula of (a) is as follows:

wherein x represents a heart rate signal data set acquired by a physiological sensor, y represents a heart rate signal data set acquired by a human face video, Cov (x, y) represents covariance of x and y, Var (x) represents variance of x, and x represents variance of x_iRepresenting the heart rate signal, y, acquired by the ith physiological sensor_iRepresenting the heart rate signal obtained for the ith personal video.

Preferably, the heart rate value extracted in step S3 is specifically:

randomly selecting partial data in the data set, calculating a heart rate value by using a human face video estimation method, and calculating beta according to the step S302₀And beta₁The remaining data is used as a prediction estimate, and the fitting effect is evaluated by using the goodness-of-fit index.

Preferably, the goodness-of-fit indicator R²The formula is as follows:

where Var (y) represents the variance of y.

Compared with the prior art, the invention has the beneficial effects that:

the heart rate value detected by the face video can be corrected according to the acquired heart rate data set, the influence of light, shaking and other factors on the detection result is overcome, and the accuracy and the reliability of the face video heart rate detection method are improved.

Drawings

Fig. 1 is a flowchart of a method for detecting a heart rate of a face video using a linear regression model according to this embodiment.

Detailed Description

The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings, and the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by persons of ordinary skill in the art without any creative effort belong to the protection scope of the present invention.

The present invention will be described in further detail with reference to fig. 1 of an embodiment of the present invention. The invention introduces a human face video heart rate detection method by using a linear regression model, which comprises the following steps:

s1, based on the iPG technology, the light irradiates the artery through the skin tissue and is reflected back to the receiving end, the obtained light intensity change rate and the blood volume change rate are in a certain relation, the relation is as follows,

wherein the content of the first and second substances,for indicating the rate of change of blood volume, Δ V_aRepresenting blood volumeVariation, V_aRepresents the volume of blood;for indicating the rate of change of light intensity,. DELTA.I indicates the change of illumination intensity,. I indicates the illumination intensity,. I₀Indicating the intensity of the illumination.

The method comprises the steps of obtaining a face image video, detecting a face region of interest (ROI), and separating three independent signal sources from a three-primary-color channel observation signal of the ROI of the face video image by using a FastICA algorithm. The specific implementation steps are as follows:

step S101, acquiring a face image video with a standard length of 30S by using a high-definition visible light camera at a good light position, keeping the acquired video relatively still, acquiring a heart rate signal by using a physiological sensor at the same time, wherein the sampling frequency is 15 frames per second and is about 450 frames, acquiring ten subjects, and acquiring 200 video segments with the same time length, detecting the positions of face key points in the face video image by using multitask convolution, and selecting a rectangular region below two eyes of a face as a face region of interest (ROI).

Step S201, separating R, G, B three primary color channels of ROI area in each frame, taking the average value of all three components of pixels in ROI area, and forming three average values of each frame into three time-series observation signals x₀(t)、x₁(t)、x₂(t) separating three independent source signals from three observed signals using the FastICA method

Assume that there are three fundamental source signals, set to s₀(t)、s₁(t)、s₂(t), the observed signal X (t) is a linear combination of the source signals S (t), i.e.

X(t)＝A·S(t)

Where A is the mixing matrix. By finding the separation matrix W to be approximately equal to A^-1So that y (t) ═ W x (t) approximates the source signal. The FastICA algorithm maximizes the objective function by taking the negative entropy of the mixed signal as the objective function, which is as follows:

J(w)＝[E{G(w^TZ)}-E{G(V)}]²

wherein G (X) ═ isX³；

S2, because the three separated independent source signals are not sequentially divided, the difference of the three independent source signals cannot be directly observed from the time domain, the three separated independent source signals are converted into frequency domain signals through FFT, the source signal with the maximum power spectrum amplitude is selected as a pulse source signal, and a heart rate value is calculated, wherein the FFT conversion formula is as follows:

selecting a source signal with the maximum power spectrum amplitude from the three source signals as a pulse source signal, and calculating a current heart rate value according to a frequency value corresponding to the maximum peak value of the frequency spectrum of the pulse source signal, wherein the heart rate calculation formula is as follows:

HR＝F_max*60

wherein, F_maxIs the frequency corresponding to the maximum peak.

And S3, creating a heart rate data set by using the heart rate of the face video collected in the step S1 and the heart rate collected by the physiological sensor, and correcting the heart rate value extracted from the face video signal by using a linear regression model. The method comprises the following concrete steps:

step S301, calculating a heart rate value according to 200 sections of human face video images acquired in step 1 by using step S1 and step S2, and acquiring a heart rate signal by using a physiological sensor, wherein the heart rate signal acquired by the physiological sensor is used as a dependent variable y_iThe heart rate signal obtained based on the face video is used as an explanatory variable x_iThe collected data set isWherein n is the number of data sets.

Step S302, a linear regression model is constructed, and the model form is as follows:

y_i＝β₀+β₁x_i+ε_i

wherein i is 1, …, n, beta₀Representing the intercept of a linear regression model，β₁Denotes the slope, ε_iIndicating an error.

Assuming the error expectation for all i is 0, using ordinary least squares, a set of β's is found₀And beta₁And minimizing the sum of the squares of the residuals, namely solving the following steps:

β₀and beta₁The calculation formula of (a) is as follows:

step S303, randomly selecting 70% of data in the data set, calculating a heart rate value by using a human face video estimation method, and calculating beta according to the step S302₀And beta₁Using the remaining 30% as a prediction estimate, using a goodness of fit indicator R²To evaluate the fitting effect, R²Ranges between 0 and 1, and the formula is as follows:

cov (x, y) denotes the covariance of x, y, Var (x) denotes the variance of x, and Var (y) denotes the variance of y.

According to the method, the heart rate data set is constructed, and the linear regression model is used for correcting the heart rate value obtained by the face video. The alignment using the linear regression model with no linear regression model is shown in table 1:

TABLE 1 comparison of heart rate values with Wireless regression model

Calculating to obtain a goodness-of-fit indicator R²The degree of fit was found to be high as 0.89. From the comparison results in table 1, the average deviation value, the standard deviation, and the like of the heart rate detection result corrected by the linear regression model are obviously reduced, so that the scheme has an obvious effect on improving the heart rate measurement accuracy, and the effectiveness of the method provided by the invention is verified.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.