阅读说明：本技术 一种脉搏波提取方法、装置、电子设备及存储介质 (Pulse wave extraction method and device, electronic equipment and storage medium ) 是由 郑秀娟 张畅 陈昱衡 杨晓梅 于 2021-09-30 设计创作，主要内容包括：本申请提供一种脉搏波提取方法、装置、电子设备及存储介质。该方法包括：获取目标视频；其中,所述目标视频中包含有待检测对象；对所述目标视频进行提取,获得初始血容量脉冲BVP信号；对所述初始BVP信号进行截取,获得子信号；根据自适应chirp模式分解算法将所述子信号进行分解,得到多个模态函数分量IMF；对所述多个IMF进行频域分析,获得所述待检测对象的脉搏波信号。本申请实施例中,通过自适应chirp模式分解算法对子信号进行分解,获得的IMF对噪声有足够的稳健性,能够很好地分离子信号与噪声的模态函数分量,使得脉搏波重构算法具有抗干扰能力,从而进一步提高提取脉搏波信号的准确性。(The application provides a pulse wave extraction method, a pulse wave extraction device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target video; the target video comprises an object to be detected; extracting the target video to obtain an initial blood volume pulse BVP signal; intercepting the initial BVP signal to obtain a sub-signal; decomposing the subsignals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF; and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected. In the embodiment of the application, the sub-signals are decomposed through the self-adaptive chirp mode decomposition algorithm, the obtained IMF has enough robustness to noise, and the modal function components of the sub-signals and the noise can be well separated, so that the pulse wave reconstruction algorithm has the anti-interference capability, and the accuracy of extracting the pulse wave signals is further improved.)

1. A pulse wave extraction method, characterized by comprising:

acquiring a target video; the target video comprises an object to be detected;

extracting the target video to obtain an initial blood volume pulse BVP signal;

intercepting the initial BVP signal to obtain a sub-signal;

decomposing the subsignals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF;

and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

2. The method according to claim 1, wherein said extracting said target video to obtain an initial blood volume pulse BVP signal comprises:

amplifying the target video according to an Euler video amplification algorithm to obtain an enhanced video;

extracting an interested area containing the object to be detected in each frame of image corresponding to the enhanced video;

and extracting a signal from a region of interest corresponding to each frame of image to obtain the initial BVP signal.

3. The method according to claim 2, wherein the extracting the signal of the region of interest corresponding to each frame of image to obtain the initial BVP signal comprises:

calculating the pixel average value of a green channel of the region of interest corresponding to each frame of image;

normalizing the pixel average value to obtain an original BVP signal;

and performing time domain filtering on the original BVP signal to obtain the original BVP signal.

4. The method of claim 1, wherein the truncating the BVP signal to obtain a sub-signal comprises:

intercepting a signal to be determined from the initial BVP signal according to a preset window width;

judging whether the signal to be determined is the sub-signal according to the following steps, and when the signal to be determined is not the sub-signal, intercepting a new signal to be determined from the initial BVP signal according to the preset window width and the preset step length until the sub-signal is acquired:

extracting a peak value feature group, a valley value feature group, a peak interval feature group, a valley interval feature group and a peak-valley difference feature group of the signal to be determined;

judging whether elements in the peak value feature group, the valley value feature group, the peak distance feature group, the valley distance feature group and the peak-valley difference feature group are in the range of the corresponding preset threshold value;

if the elements in each feature group are within the corresponding preset threshold value range, determining the signal to be determined as the sub-signal;

and if the characteristic group contains elements which do not meet the corresponding preset threshold value, the signal to be determined is not the sub-signal.

5. The method according to claim 4, wherein the determining whether the elements in the peak feature set, the valley feature set, the peak-to-pitch feature set, the valley-to-pitch feature set, and the peak-to-valley difference feature set are within the respective preset threshold ranges comprises:

calculating a first threshold range corresponding to a peak value according to elements of the peak value feature group, calculating a second threshold range corresponding to a valley value according to elements of the valley value feature group, calculating a third threshold range corresponding to a peak distance according to elements of the peak distance feature group, calculating a fourth threshold range corresponding to a valley distance according to elements of the valley distance feature group and calculating a fifth threshold range corresponding to an element of a peak-valley difference feature value;

and judging whether the elements of the peak feature group are in the first threshold range, whether the elements of the valley feature group are in the second threshold range, whether the elements of the peak-pitch feature group are in the third threshold range, whether the elements of the valley-pitch feature group are in the fourth threshold range and whether the elements of the peak-valley difference feature group are in the fifth threshold range.

6. The method according to any one of claims 1-5, wherein the performing a frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the subject to be detected comprises:

performing frequency domain analysis on the plurality of IMFs to obtain main frequencies corresponding to the IMFs respectively;

calculating a correlation coefficient for each of the IMFs and the sub-signals;

and determining the IMF of which the main frequency is in a preset frequency range and the maximum value of the correlation coefficient corresponds to the main frequency as a pulse wave signal.

7. The method of claim 6, further comprising:

acquiring the number of peak values of the pulse wave signals;

and determining a heart rate value according to the number of the peak values.

8. A pulse wave extraction device characterized by comprising:

the acquisition module is used for acquiring a target video; the target video comprises an object to be detected;

the acquisition module is used for extracting the target video to acquire an initial blood volume pulse BVP signal;

the intercepting module is used for intercepting the BVP signal to obtain a sub-signal;

the decomposition module is used for decomposing the sub-signals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF;

and the frequency domain analysis module is used for carrying out frequency domain analysis on the IMFs to obtain the pulse wave signals of the object to be detected.

9. An electronic device, comprising: a processor, a memory, and a bus, wherein,

the processor and the memory are communicated with each other through the bus;

the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1-7.

Technical Field

The application relates to the field of biomedicine, in particular to a pulse wave extraction method, a pulse wave extraction device, electronic equipment and a storage medium.

Background

At present, pulse waves are measured in a contact mode such as a finger clip, special data measuring equipment is needed, and meanwhile, the contact type measuring method has special situations which are not suitable for use, for example, the measurement of infants or burn patients has certain difficulty.

In the field of biomedicine, the limitation of the application of a contact type pulse wave calculation physiological parameter scene is solved through non-contact type pulse wave measurement. The imaging type photoelectric volume pulse wave technology is based on an LED light source and a detector, measures attenuation light reflected and absorbed by blood vessels and tissues of a human body, records the pulsation state of the blood vessels and measures pulse waves.

However, due to the influence of the acquisition environment and the motion state of the measurer, the initial Blood Volume Pulse (BVP) signal of the measurement is easily interfered by the artifact, so that the obtained initial BVP is a non-stationary and non-linear signal. Empirical Mode Decomposition (EMD) is a method for processing nonlinear and non-stationary signals, and EMD gradually decomposes signals into Intrinsic Mode Functions (IMFs) by using different characteristic scales, and selects partial IMF components to reconstruct signals to achieve the effect of denoising, but the EMD method has the problem of signal aliasing. The Variable Mode Decomposition (VMD) algorithm solves the problem of signal aliasing caused by EMD, but VMD does not work well when dealing with multi-component signals with overlapping frequencies.

Disclosure of Invention

An object of the embodiments of the present application is to provide a pulse wave extraction method, an apparatus, an electronic device, and a storage medium, for solving a technical problem in the prior art that an extracted pulse wave is not good in effect due to noise and interference in a reconstructed pulse wave signal.

In a first aspect, an embodiment of the present application provides a pulse wave extraction method, including obtaining a target video; the target video comprises an object to be detected; extracting the target video to obtain an initial blood volume pulse BVP signal; intercepting the initial BVP signal to obtain a sub-signal; decomposing the subsignals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF; and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

In the embodiment of the application, modal decomposition is performed on an initial BVP signal through Adaptive Chirp Mode Decomposition (ACMD), the problem that the effect of processing frequency-overlapped multi-component signals by a variational modal decomposition algorithm is poor is solved, time domain and frequency domain resolution are improved, noise signals can be preprocessed better, the obtained IMF has enough robustness on noise, signals and noise modal function components can be well separated, the pulse wave reconstruction algorithm has anti-interference capacity, and the accuracy of extracted pulse wave signals is improved.

Optionally, in this embodiment of the present application, extracting the target video to obtain an initial blood volume pulse BVP signal includes:

amplifying the target video according to an Euler video amplification algorithm to obtain an enhanced video; extracting an interested area containing the object to be detected in each frame of image corresponding to the enhanced video; and extracting a signal from a region of interest corresponding to each frame of image to obtain the initial BVP signal.

In the embodiment of the application, the target video is amplified by using the Euler video amplification algorithm, so that the interference of ambient light change resistance is facilitated, meanwhile, the noise cannot be amplified when the target video is amplified, a better amplification effect is achieved, signals of an interested area are extracted, and the interference of motion interference and shooting background can be avoided.

Optionally, in this embodiment of the present application, the performing signal extraction on the region of interest corresponding to each frame of image to obtain the initial BVP signal includes:

calculating the pixel average value of a green channel of the region of interest corresponding to each frame of image; normalizing the pixel average value to obtain an original BVP signal; and performing time domain filtering on the original BVP signal to obtain the original BVP signal.

In the embodiment of the application, the video signal can be converted into the time signal by calculating the pixel average value of the green channel, the time domain filtering is carried out on the original BVP signal, and only the signal within the range of [0.5, 4] Hz is reserved, so that the resource waste caused by the follow-up processing of the irrelevant frequency signal is avoided.

Optionally, intercepting the BVP signal to obtain a sub-signal includes:

intercepting a signal to be determined from the initial BVP signal according to a preset window width; judging whether the signal to be determined is the sub-signal according to the following steps, and when the signal to be determined is not the sub-signal, intercepting a new signal to be determined from the initial BVP signal according to the preset window width and the preset step length until the sub-signal is acquired:

extracting a peak value feature group, a valley value feature group, a peak interval feature group, a valley interval feature group and a peak-valley difference feature group of the signal to be determined; judging whether elements in the peak value feature group, the valley value feature group, the peak distance feature group, the valley distance feature group and the peak-valley difference feature group are in the range of the corresponding preset threshold value; if the elements in each feature group are within the corresponding preset threshold value range, determining the signal to be determined as the sub-signal; and if the characteristic group contains elements which do not meet the corresponding preset threshold value, the signal to be determined is not the sub-signal.

In the embodiment of the application, the waveform segment interfered by movement is removed by carrying out abnormity judgment on elements of a peak value feature group, a valley value feature group, a peak distance feature group, a valley distance feature group and a peak-valley difference value feature group of the sub-signals, and the effectiveness of the initial BVP signal for subsequent pulse wave extraction is ensured.

Optionally, the determining whether elements in the peak feature group, the valley feature group, the peak-to-peak feature group, the valley-to-valley feature group, and the peak-to-valley difference feature group are within respective corresponding preset threshold ranges includes:

calculating a first threshold range corresponding to a peak value according to elements of the peak value feature group, calculating a second threshold range corresponding to a valley value according to elements of the valley value feature group, calculating a third threshold range corresponding to a peak distance according to elements of the peak distance feature group, calculating a fourth threshold range corresponding to a valley distance according to elements of the valley distance feature group and calculating a fifth threshold range corresponding to an element of a peak-valley difference feature value;

and judging whether the elements of the peak feature group are in the first threshold range, whether the elements of the valley feature group are in the second threshold range, whether the elements of the peak-pitch feature group are in the third threshold range, whether the elements of the valley-pitch feature group are in the fourth threshold range and whether the elements of the peak-valley difference feature group are in the fifth threshold range.

In the embodiment of the application, whether abnormal data exist in the elements of the feature group is judged through the boxed graph theory, so that the abnormal waveform can be still accurately judged under the condition that the elements of the feature group do not meet normal distribution, and the robustness of the judgment result is guaranteed.

Optionally, the performing frequency domain analysis on the plurality of IMFs to obtain the pulse wave signal of the object to be detected includes: performing frequency domain analysis on the plurality of IMFs to obtain main frequencies corresponding to the IMFs respectively; calculating a correlation coefficient for each of the IMFs and the sub-signals; and determining the IMF of which the main frequency is in a preset frequency range and the maximum value of the correlation coefficient corresponds to the main frequency as a pulse wave signal.

In the embodiment of the application, the IMF components are subjected to frequency domain analysis, the main frequency is determined to be within the preset frequency range, and the IMF corresponding to the maximum value of the correlation coefficient is used as the pulse wave signal, so that the accuracy of the extracted pulse wave signal is improved.

Optionally, the method further includes: acquiring the number of peak values of the pulse wave signals; and determining a heart rate value according to the number of the peak values.

In the embodiment of the application, the size of heart rate can be obtained according to the pulse wave signal that obtains, through the accuracy that improves the pulse wave signal that draws for physiological parameters such as the heart rate value that obtains are more accurate, provide certain reference for daily life non-contact heart rate estimation.

In a second aspect, an embodiment of the present application provides a pulse wave extraction apparatus, including: the acquisition module is used for acquiring a target video; the target video comprises an object to be detected; the acquisition module is used for extracting the target video to acquire an initial blood volume pulse BVP signal; the intercepting module is used for intercepting the BVP signal to obtain a sub-signal; the decomposition module is used for decomposing the sub-signals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF; and the frequency domain analysis module is used for carrying out frequency domain analysis on the IMFs to obtain the pulse wave signals of the object to be detected.

In a third aspect, an embodiment of the present application provides an electronic device, including: the system comprises a processor, a memory and a bus, wherein the processor and the memory are communicated with each other through the bus; the memory stores program instructions executable by the processor, the processor being capable of performing the method of the first aspect when invoked by the program instructions.

In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, including: the non-transitory computer readable storage medium stores computer instructions that cause the computer to perform the method of the first aspect.

Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a schematic flow chart of a pulse wave extraction method according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram illustrating characteristics of a signal quality detection algorithm provided in an embodiment of the present application;

FIG. 3 is a flow chart of a quality detection algorithm provided by an embodiment of the present application;

fig. 4 is a schematic waveform diagram of a pulse wave signal and a real signal provided in the present embodiment;

fig. 5 is a schematic structural diagram of a pulse wave extracting apparatus according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

Fig. 1 is a schematic flowchart of a pulse wave extraction method provided in an embodiment of the present application, where the method can be applied to a terminal device (also referred to as an electronic device) and a server; the terminal device may be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or the like; the server may specifically be an application server, and may also be a Web server.

The method comprises the following steps:

step 101: acquiring a target video; the target video comprises an object to be detected.

In a specific implementation process, the obtaining manner of the target video in step 101 includes, but is not limited to, the following: the first acquisition mode is that a target object is shot by using acquisition equipment such as a video camera, a video recorder or a color camera to obtain a target video, then the acquisition equipment sends the target video to terminal equipment, then the terminal equipment receives the target video sent by the acquisition equipment, and the terminal equipment can store the target video into a file system, a database or mobile storage equipment; the second obtaining method is to obtain a pre-stored target video, and specifically includes: acquiring a target video from a file system, or acquiring a video to be targeted from a database, or acquiring the target video from a mobile storage device; and the third acquisition mode is to acquire a target video on the internet by using software such as a browser or the like, or to access the internet by using other application programs to acquire the target video. Wherein, the object to be detected refers to the naked skin part of the human body.

Step 102: and extracting the target video to obtain an initial blood volume pulse BVP signal.

The initial BVP signal is an initial blood volume pulse signal, and during the acquisition process, the interference of artifact signals, such as the interference of irregular flash, natural light or motion artifacts, can be obtained, so that the obtained BVP signal is a non-stationary and non-linear signal, therefore, before the BVP signal is extracted, the target signal is preprocessed, including amplification and denoising, and then the initial BVP signal is extracted, so that the waveform segment affected by motion interference can be removed, and the effectiveness of the initial BVP signal for subsequent pulse wave extraction is ensured.

Step 103: and intercepting the initial BVP signal to obtain a sub-signal.

The initial BVP signal can be intercepted through a window function, a signal with a required width is obtained by setting the window width and the step length of the window function, and the sub-signal is a signal with a specific width obtained by intercepting the initial BVP signal.

Step 104: and decomposing the subsignals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF.

The self-adaptive chirp mode decomposition algorithm is a non-stationary signal processing method based on variational modal decomposition, a sub-signal is decomposed into a plurality of modal components with certain bandwidth and energy from large to small, the modal function component IMF is a local characteristic signal containing different time scales of the sub-signal, the instantaneous frequency of any point is meaningful, any signal is composed of a plurality of eigenmode functions, at any time, one signal can contain a plurality of eigenmode functions, and if the eigenmode functions are mutually overlapped, a composite signal is formed.

In the embodiment of the application, the sub-signals are sequentially decomposed into K modal components with certain bandwidth and energy from large to small according to the energy of the sub-signals by the self-adaptive chirp pattern decomposition algorithm, the self-adaptive chirp pattern decomposition algorithm is optimized by the greedy algorithm, a mathematical model is established for the modal components obtained by decomposition to describe the problem, the solved problem is divided into a plurality of sub-problems, each sub-problem is solved to obtain the local optimal solution of the sub-problem, and the local optimal solution of the sub-problem is synthesized into a whole solution.

The specific numerical value of K is obtained in a self-adaptive mode through a preset energy threshold, when the energy of the modal components obtained through decomposition is lower than the preset energy threshold, decomposition is not carried out, and the number of the modal components obtained through decomposition is the number of the self-adaptive modal components.

The process of decomposing the sub-signals to obtain the ith IMF is converted into a problem of solving an optimal solution, wherein:

in the above formula, α is a weight coefficient, θ_iFor the ith demodulated signal or b_iPhase of (t), f_i(t) is a frequency equation,for the estimated frequency equation, s_i(t) is the i-th modal component obtained by decomposition, s can be expressed by_i(t) is regarded as a modulation signal, a_i(t) and b_i(t) is regarded as two demodulated signals, A_i(t) is the amplitude.

Step 105: and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

The time-domain performance of the IMF can be indirectly revealed by performing frequency-domain analysis on the IMF, a spectrogram of each IMF is obtained by performing frequency-domain analysis, the main frequency of each IMF component can be obtained by analyzing the spectrogram, the IMF component with the main frequency within the range of [0.5, 4] Hz and the maximum correlation with the sub-signal in the step 103 is selected as a reconstructed BVP signal, namely, a human pulse wave signal is obtained.

On the basis of the above embodiment, the extracting the target video to obtain the initial blood volume pulse BVP signal includes:

amplifying the target video according to an Euler video amplification algorithm to obtain an enhanced video;

extracting an interested area containing the object to be detected in each frame of image corresponding to the enhanced video;

and extracting a signal from a region of interest corresponding to each frame of image to obtain the initial BVP signal.

The key steps of the Eulerian Video amplification (EVM) algorithm include:

taking the collected target video as input;

carrying out pyramid multi-resolution decomposition on the target video, and decomposing each frame image of the input target video into a Gaussian pyramid mode;

performing time domain band-pass filtering on each frame image to obtain a plurality of interested frequency bands;

carrying out Taylor series differential approximation on the signal of each frequency band to obtain the effect of linear amplification approximation;

the signal of each frequency band is reconstructed to obtain a color-amplified video, i.e., an enhanced video.

The EVM analyzes the time-space change characteristics of video contents from the global angle, senses the tiny change of an image sequence, realizes the amplification and visualization of the color change or tiny movement of an interested frequency segment, reduces the influence on the amplification and strength shearing artifacts of noise, and has the inhibiting effect on the change influence of environmental illumination and the like.

In the embodiment of the application, a human face part is taken as an example, in order to avoid motion interference caused by eyes and a mouth, a human face detection algorithm and a human face characteristic point marking algorithm are adopted to perform human face detection and characteristic point marking on each frame of an enhanced video, a forehead is divided into an interested area according to human face characteristic points, interference of head motion and shooting background is avoided, and the interested area can be adjusted according to the motion condition of the face.

On the basis of the above embodiment, the performing signal extraction on the region of interest corresponding to each frame of image to obtain the initial BVP signal includes:

calculating the pixel average value of a green channel of the region of interest corresponding to each frame of image;

normalizing the pixel average value to obtain an original BVP signal;

and performing time domain filtering on the original BVP signal to obtain the original BVP signal.

In the embodiment of the application, the system divides the forehead as an interested area according to the human face characteristic points, calculates the pixel average value of R, G, B channels of the interested area corresponding to each frame of image, wherein the signal regularity of the red channel and the blue channel of the forehead area is poor, the waveform of the pulse wave is not easy to extract, and the green channel signal has a certain regularity, so the pixel average value of the green channel is selected as the input signal of the next stage, and the pixel average value of the green channel is normalized to the range of [ -1,1] to obtain the original BVP signal.

Because the normal pulse frequency range of the human body is 30 times/minute-120 times/minute, the band-pass range of the Butterworth filter is set to [0.5, 4] Hz, the initial BVP signal is subjected to time-domain filtering, the initial BVP signal is converted into the [0.5, 4] Hz range, and the initial BVP signal is obtained.

On the basis of the foregoing embodiment, the intercepting the BVP signal to obtain a sub-signal includes:

intercepting a signal to be determined from the initial BVP signal according to a preset window width;

judging whether the signal to be determined is the sub-signal according to the following steps, and when the signal to be determined is not the sub-signal, intercepting a new signal to be determined from the initial BVP signal according to the preset window width and the preset step length until the sub-signal is acquired:

extracting a peak value feature group, a valley value feature group, a peak interval feature group, a valley interval feature group and a peak-valley difference feature group of the signal to be determined;

judging whether elements in the peak value feature group, the valley value feature group, the peak distance feature group, the valley distance feature group and the peak-valley difference feature group are in the range of the corresponding preset threshold value;

if the elements in each feature group are within the corresponding preset threshold value range, determining the signal to be determined as the sub-signal;

and if the characteristic group contains elements which do not meet the corresponding preset threshold value, the signal to be determined is not the sub-signal.

In the embodiment of the application, a peak characteristic group, a valley characteristic group, a peak-to-valley distance characteristic group, a valley-to-valley distance characteristic group and a peak-to-valley difference characteristic group of a signal to be determined are extracted, and the elements of the five characteristic groups of the sub-signal to be determined are subjected to abnormity judgment. When the abnormal value of the data is judged, on the basis of analyzing the mean value and the standard of the data, whether the data contains the abnormal value can be calculated by using the 3 alpha rule, but the 3 alpha rule requires that the data obeys normal distribution, and the data which does not meet the normal distribution is not suitable for the 3 alpha rule.

In the embodiment of the application, data are judged abnormally through a box type graph theory, the box type graph is a graph capable of describing data positions and dispersion conditions and is still suitable for data which do not meet normal distribution, and the essence of the box type graph is thatDividing data into four regions with the dividing point being the upper limit Q₄Upper quartile Q₃Median Q₂Lower quartile Q₁Lower limit of Q₀The normal data is distributed at the upper limit Q₄And the lower limit Q₀While the anomalous data is at the upper limit Q₄And lower limit Q₀And (c) out.

The calculation formula is as follows:

Q₃＝N_{((n+1)×3/4-1})×0.75+N_{((n+1)×3/4+1)}×0.25

Q₁＝N_((n+1)/4-1)×0.25+N_((n+1)/4+1)×0.75

Q₄＝Q₃+1.5×(Q₃-Q₁)

Q₀＝Q₁-1.5×(Q₃-Q₁)

wherein Q is₂The number of the sequences is an even number, the median of the group is the average of the two middle numbers, and n is the array length of the characteristic group for carrying out abnormity judgment.

Fig. 2 is a schematic diagram illustrating characteristics of a signal quality detection algorithm according to an embodiment of the present application. In general, there will be 15 peaks and valleys, 14 peak-to-valley distances and valley-to-valley distances, 29 peak-to-valley differences within 0-15 seconds, and FIG. 2 shows the extracted features as 5 peaks and valleys, 4 peak-to-valley distances and 9 peak-to-valley differences, where the horizontal axis is time 0-15 seconds and the vertical axis is the amplitude of the sub-signal, where P is the amplitude of the sub-signal₁、P₂、P₃、P₄、P₅And P₆Is the peak value corresponding to the sub-signal within 0-15 seconds, v₁、v₂、v₃、v₄、v₅And v₆Is the valley value dp corresponding to the sub-signal within 0-15 seconds₁、dp₂、dp₃、dp₄And dp₅The corresponding peak distance, dv, of the sub-signal in 0-15 seconds₁、dv₂、dv₃、dv₄And dv₅The valley pitch, dpv, corresponding to the sub-signal in 0-15 seconds₁、dpv₂、dpv₃、dpv₄、dpv₅、dpv₆、dpv₇、dpv₈、dpv₉、dpv₁₀And dpv₁₁The peak-to-valley difference corresponding to the sub-signals in 0-15 seconds.

In the embodiment of the application, the extracted 15-second signal to be determined is taken as an example, the quality detection is carried out on the 15-second signal, and the peak characteristic group of the 15-second signal is extractedValley value characteristic set Peak pitch feature setValley spacing feature setSum peak to valley difference feature set

In the embodiment of the application, the peak characteristic group is used For example, the data is judged by the boxplot theory.

Arranging the elements of the peak characteristic group from small to large to obtain The following can be obtained by calculation:

lower quartile Q₁＝0.25×N₄+0.75×N₄12.2, wherein N₄Is an ordered set of featuresThe fourth item of (1);

upper quartile Q₃＝0.25×N₁₂+0.75×N₁₂0.25 × 13.0+0.75 × 13.0 ═ 13.0, where N is₁₂Is an ordered set of featuresThe twelfth item of (1);

lower limit Q₀＝Q₁-1.5×(Q₃-Q₁)＝11；

Upper limit of Q₄＝Q₃+1.5×(Q₃-Q₁)＝14.2；

Median Q₂＝0.5×N₈+0.5×N₈0.5 × 12.4+0.5 × 12.4 ═ 12.4, where N is₈Is an ordered set of featuresThe eighth item of (1).

And (4) judging the elements of the peak characteristic group and [11, 14.2], and judging that the elements of the peak characteristic group have no abnormal data if the elements of the obtained peak characteristic group are all in the threshold range.

In this embodiment of the present application, it is to be determined whether the signal to be determined is the sub-signal, and when the signal to be determined is not the sub-signal, a new signal to be determined is intercepted from the initial BVP signal according to the preset window width and the preset step length again until the sub-signal is acquired.

In order to reduce the influence of motion artifact interference on the sub-Signal decomposition, before the sub-Signal decomposition, a Signal Quality inspection (QA) algorithm is used to extract a waveform with qualified Quality for processing and analysis, and fig. 3 is a flowchart of a Quality inspection algorithm provided in the embodiment of the present application.

Firstly, an initial BVP signal is intercepted, and firstly, a preset window width is 15 seconds, a step size is 1 second, and a width threshold of a sub-signal is set to be 5.

Step 201: and acquiring a sub-signal.

And if the preset initial index is 0, the corresponding ending index is the initial index plus the window width, the ending index is 15 according to the preset initial index 0 and the window width, the intercepted signal to be determined is judged to be an effective signal if the width of the intercepted signal to be determined is more than 5, and the sub-signal of 0-15s is intercepted.

Step 202: and carrying out abnormity judgment on the intercepted signal to be determined.

Firstly, performing feature conversion on a signal to be determined, extracting a peak feature group, a valley feature group, a peak-to-peak distance feature group, a valley-to-valley distance feature group and a peak-to-valley difference feature group of a sub-signal to be determined, respectively calculating a threshold range corresponding to each feature group according to elements of each feature group, and judging whether the elements of each feature group are in the corresponding threshold range according to the threshold range corresponding to each feature group.

And if the elements in each feature group are within the corresponding preset threshold value range, determining the signal to be determined as the sub-signal, and extracting the sub-signal for reconstructing the pulse wave signal.

Step 203: and if the characteristic group contains elements which do not meet the corresponding preset threshold value, judging whether the signal to be determined is not the sub-signal, and judging whether the ending index of the signal to be determined is larger than the length of the original signal S.

If the ending index is smaller than the length of the original signal S, adjusting a starting index and an ending index, wherein the starting index is the original starting index plus the step length, the corresponding ending index is the starting index plus the window width, intercepting a new signal to be determined, and judging whether the signal to be determined contains an abnormal value or not until a sub-signal without the abnormal value is extracted.

If the ending index is greater than the length of the original signal S, changing the preset window width into the original window width-step length, then re-acquiring a corresponding ending index according to the preset window width, judging whether the length of the signal to be determined acquired according to the current starting index and the ending index is greater than a width threshold value 5, if the length of the signal to be determined is greater than the width threshold value 5, intercepting the signal to be determined to perform subsequent abnormal judgment, if the length of the signal to be determined is less than the width threshold value 5, indicating that no sub-signal which does not contain abnormal data exists in the initial BVP signal, re-extracting the initial BVP signal, and then performing quality detection.

On the basis of the above embodiment, performing frequency domain analysis on the plurality of IMFs to obtain a pulse wave signal of the object to be detected includes:

performing frequency domain analysis on the plurality of IMFs to obtain main frequencies corresponding to the IMFs respectively; calculating a correlation coefficient for each of the IMFs and the sub-signals; and determining the IMF of which the main frequency is in a preset frequency range and the maximum value of the correlation coefficient corresponds to the main frequency as a pulse wave signal.

And performing frequency domain analysis on each IMF to obtain a corresponding spectrogram, and taking the frequency corresponding to the maximum peak value of the IMF in the spectrogram as the main frequency of each IMF component. Meanwhile, a correlation coefficient between each IMF and the sub-signal obtained in step 103 is calculated, an IMF component having a main frequency in the range of [0.5, 4] Hz and a maximum correlation with the sub-signal before decomposition is selected as a reconstructed BVP signal, and a human pulse wave signal is obtained.

In the embodiment of the application, modal decomposition is carried out on the initial BVP signal through self-adaptive chirp mode decomposition, the problem that the effect of multi-component signals with overlapped frequencies processed by a variational modal decomposition algorithm is poor is solved, the time domain and frequency domain resolution are improved, noise signals can be better preprocessed, the obtained IMF has enough robustness on noise, signals and noise modal function components can be well separated, the pulse wave reconstruction algorithm has anti-interference capacity, and the accuracy of extracted pulse wave signals is improved.

Fig. 4 is a schematic waveform diagram of a pulse wave signal and a real signal provided in the present application, in which a dotted line is a waveform of the real signal, and a solid line is a reconstructed pulse wave signal. And calculating a heart rate value by adopting a peak detection algorithm, and determining the heart rate value according to the number of peaks of the reconstructed BVP signal waveform diagram, wherein for example, if a signal waveform diagram with the duration of 30 seconds has 35 peaks, the corresponding heart rate value is 70 times/minute.

Under five tested environments of low illumination, normal environment, high illumination, unbalanced illumination and head shaking, corresponding pulse wave signals are respectively extracted through an Empirical Mode Decomposition (EMD) algorithm, a Variational Mode Decomposition (VMD) algorithm, an Adaptive Chirp Mode Decomposition (ACMD) algorithm, an Euler video amplification algorithm-an empirical mode decomposition algorithm (EVM _ EMD), an Euler video amplification algorithm-a variational mode decomposition algorithm (EVM _ VMD), an Euler video amplification algorithm-an adaptive chirp mode decomposition algorithm (EVM _ ACMD) and an Euler video amplification algorithm-a quality detection algorithm-an adaptive chirp mode decomposition algorithm (EVM _ QA _ ACMD), and the Pearson correlation coefficient of each pulse wave signal and each standard signal is calculated.

Table 1 shows the pearson correlation coefficient between the reconstructed pulse wave signal and the real signal provided in the embodiment of the present application, and the correlation coefficient between the pulse wave signal obtained by EVM _ QA _ ACMD and the standard signal obtained in table 1 is the highest in all five tested environments, 0.85 in a low light environment, 0.87 in a normal environment, 0.86 in a high light environment, 0.84 in an unbalanced light environment, and 0.85 in a head shaking condition.

TABLE 1

Through the data analysis, the change of fine signals can be enhanced through an Euler video amplification algorithm EVM, a signal quality detection algorithm QA can effectively screen out a waveform segment with qualified quality, and an ACMD algorithm separates effective signals related to heart rate from noise signals, so that the EVM _ QA _ ACMD can ensure the consistency of estimated pulse wave signals and standard signals to a certain extent, and further can ensure the accurate calculation of physiological parameters such as heart rate values and the like.

The BVP signal reconstructed by EVM _ QA _ ACMD is used for calculating a heart rate value, and the evaluation indexes comprise: mean Absolute Error (MAE), Standard Deviation (SD), Pearson correlation coefficient (Pcc).

The calculation formulas of MAE, SD and Pcc are as follows:

wherein R is_iIs the ith heart rate estimate, G_iFor the ith heart rate true value, N is the total number of samples calculated for the heart rate, and COV (R, G) is a covariance function of the heart rate estimate and the true value.

Table 2 shows statistical indicators of heart rate estimation under different scenarios provided in the embodiment of the present application, and according to the calculation data, the average absolute error between the heart rate estimation value and the true value obtained by the EVM _ QA _ ACMD method under the low illumination environment is 5.14bpm, the standard deviation is 5.64bpm, and the pearson correlation coefficient is 0.73; under the normal illumination environment, the average absolute error is 4.32bpm, the standard deviation is 4.55bpm, and the Pearson correlation coefficient is 0.94; under the high illumination environment, the average absolute error is 3.65bpm, the standard deviation is 3.54, and the Pearson correlation coefficient is 0.93; under the unbalanced illumination environment, the average absolute error is 5.39bpm, the standard deviation is 4.96bpm, and the Pearson correlation coefficient is 0.85; the mean absolute error in the case of head shaking was 3.75bpm, the standard deviation was 5.95bpm, and the Pearson correlation coefficient was 0.84.

TABLE 1

Through the analysis, under five test environments including a low-illumination environment, a high-illumination environment, uneven brightness of the left face and the right face, a normal-illumination environment and head shaking, the pulse wave signals obtained by combining the Euler video amplification algorithm, the quality detection algorithm and the adaptive chirp mode decomposition algorithm (EVM _ QA _ ACMD) can obtain a good heart rate estimation result, and the heart rate estimation value is better consistent with a real value.

In summary, the present application proposes for the first time that a pulse wave signal obtained by reconstructing an initial BVP signal based on an euler video amplification algorithm, a signal quality detection algorithm, and an adaptive chirp pattern decomposition algorithm is used for performing a non-contact measurement on a heart rate value, and is more stable than other methods, has higher consistency with a standard signal, and has better robustness on an illumination artifact and a motion artifact, and a heart rate value estimated by using the method can obtain an excellent effect, and provides a certain reference for non-contact heart rate estimation in daily life.

Fig. 5 is a schematic structural diagram of a pulse wave extraction apparatus according to an embodiment of the present disclosure, where the apparatus may be a module, a program segment, or a code on an electronic device. It should be understood that the apparatus corresponds to the above-mentioned embodiment of the method of fig. 1, and can perform various steps related to the embodiment of the method of fig. 1, and the specific functions of the apparatus can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy. The device includes: the device comprises an acquisition module 301, an acquisition module 302, a truncation module 303, a decomposition module 304 and a frequency domain analysis module 305, wherein:

the acquisition module 301 is used for acquiring a target video; the target video comprises an object to be detected; the acquisition module is used for extracting the target video to acquire an initial blood volume pulse BVP signal; the intercepting module is used for intercepting the BVP signal to obtain a sub-signal; the decomposition module is used for decomposing the sub-signals according to a self-adaptive chirp pattern decomposition algorithm to obtain a plurality of modal function components IMF; and the frequency domain analysis module is used for carrying out frequency domain analysis on the IMFs to obtain the pulse wave signals of the object to be detected.

On the basis of the foregoing embodiment, the obtaining module 302 is specifically configured to:

amplifying the target video according to an Euler video amplification algorithm to obtain an enhanced video; extracting an interested area containing the object to be detected in each frame of image corresponding to the enhanced video; and extracting a signal from a region of interest corresponding to each frame of image to obtain the initial BVP signal.

On the basis of the foregoing embodiment, the obtaining module 302 is specifically configured to:

calculating the pixel average value of a green channel of the region of interest corresponding to each frame of image;

normalizing the pixel average value to obtain an original BVP signal;

and performing time domain filtering on the original BVP signal to obtain the original BVP signal.

On the basis of the foregoing embodiment, the intercepting module 303 is specifically configured to:

intercepting a signal to be determined from the initial BVP signal according to a preset window width;

judging whether the signal to be determined is the sub-signal according to the following steps, and when the signal to be determined is not the sub-signal, intercepting a new signal to be determined from the initial BVP signal according to the preset window width and the preset step length until the sub-signal is acquired:

extracting a peak value feature group, a valley value feature group, a peak interval feature group, a valley interval feature group and a peak-valley difference feature group of the signal to be determined;

judging whether elements in the peak value feature group, the valley value feature group, the peak distance feature group, the valley distance feature group and the peak-valley difference feature group are in the range of the corresponding preset threshold value;

if the elements in each feature group are within the corresponding preset threshold value range, determining the signal to be determined as the sub-signal;

and if the characteristic group contains elements which do not meet the corresponding preset threshold value, the signal to be determined is not the sub-signal.

On the basis of the foregoing embodiment, the intercepting module 303 is specifically configured to:

calculating a first threshold range corresponding to a peak value according to elements of the peak value feature group, calculating a second threshold range corresponding to a valley value according to elements of the valley value feature group, calculating a third threshold range corresponding to a peak distance according to elements of the peak distance feature group, calculating a fourth threshold range corresponding to a valley distance according to elements of the valley distance feature group and calculating a fifth threshold range corresponding to an element of a peak-valley difference feature value;

and judging whether the elements of the peak feature group are in the first threshold range, whether the elements of the valley feature group are in the second threshold range, whether the elements of the peak-pitch feature group are in the third threshold range, whether the elements of the valley-pitch feature group are in the fourth threshold range and whether the elements of the peak-valley difference feature group are in the fifth threshold range.

On the basis of the foregoing embodiment, the frequency domain analysis module 305 is specifically configured to:

performing frequency domain analysis on the plurality of IMFs to obtain main frequencies corresponding to the IMFs respectively; calculating a correlation coefficient for each of the IMFs and the sub-signals; and determining the IMF of which the main frequency is in a preset frequency range and the maximum value of the correlation coefficient corresponds to the main frequency as a pulse wave signal.

On the basis of the foregoing embodiment, the apparatus further includes a peak obtaining module, configured to:

acquiring the number of peak values of the pulse wave signals; and determining a heart rate value according to the number of the peak values.

To sum up, in the embodiment of the application, realize the video enhancement through the EVM algorithm, help anti ambient light to change the interference, before carrying out signal decomposition, use signal quality detection algorithm to extract the qualified waveform of quality and handle and analyze, influence to signal decomposition in order to reduce motion artifact interference, self-adaptation chirp mode decomposition has solved the not good problem of multicomponent signal effect that VMD algorithm processing frequency overlaps simultaneously, time domain and frequency domain resolution have been improved, can be better carry out the preliminary treatment to the noise signal, the IMF who obtains has sufficient robustness to the noise, can separate signal and noise modal function component well, make pulse wave reconstruction algorithm have the interference killing feature, improve the accuracy of the pulse wave signal of extraction.

Fig. 6 is a schematic structural diagram of an entity of an electronic device provided in an embodiment of the present application, and as shown in fig. 6, the electronic device includes: a processor (processor)401, a memory (memory)402, and a bus 403; wherein:

the processor 401 and the memory 402 complete communication with each other through the bus 403;

the processor 401 is configured to call the program instructions in the memory 402 to execute the methods provided by the above-mentioned method embodiments, for example, including: acquiring a target video; the target video comprises an object to be detected; extracting the target video to obtain an initial BVP signal; intercepting the initial BVP signal to obtain a sub-signal; reconstructing the sub-signals according to a self-adaptive chirp mode decomposition algorithm to obtain a plurality of modal function components IMF; and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

The processor 401 may be an integrated circuit chip having signal processing capabilities. The Processor 401 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. Which may implement or perform the various methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The Memory 402 may include, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Read Only Memory (EPROM), Electrically Erasable Read Only Memory (EEPROM), and the like.

The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, comprising: acquiring a target video; the target video comprises an object to be detected; extracting the target video to obtain an initial BVP signal; intercepting the initial BVP signal to obtain a sub-signal; reconstructing the sub-signals according to a self-adaptive chirp mode decomposition algorithm to obtain a plurality of modal function components IMF; and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring a target video; the target video comprises an object to be detected; extracting the target video to obtain an initial BVP signal; intercepting the initial BVP signal to obtain a sub-signal; reconstructing the sub-signals according to a self-adaptive chirp mode decomposition algorithm to obtain a plurality of modal function components IMF; and carrying out frequency domain analysis on the plurality of IMFs to obtain the pulse wave signals of the object to be detected.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.