阅读说明：本技术 一种面向可穿戴传感器的人类活动识别方法 (Human activity recognition method facing wearable sensor ) 是由 马春梅 孙华志 姜丽芬 梁研 宿通通 于 2019-09-19 设计创作，主要内容包括：本发明公开了一种面向可穿戴传感器的人类活动识别方法,首先,将感知的时序异构数据形成指纹矩阵并将按滑动窗口的大小进行切分后的数据作为模型输入,然后通过由前向长短期记忆和后向长短期记忆构成的双向LSTM层处理输入数据,获得源数据的粗粒度特征,之后以注意力机制层对先前的粗粒度特征进行重要度计算,以便获得能够反映活动特性的细粒度特征,最后用分类的逻辑回归处理细粒度特征,获得当前数据的多个标签的概率分布,从而最终判定活动类型。本发明提高了可穿戴传感器对用户活动的认知能力,可以对用户活动进行精确的识别,提高人机交互能力。(The invention discloses a human activity recognition method facing a wearable sensor, which comprises the steps of firstly, forming a fingerprint matrix by perceived time sequence heterogeneous data, taking data after segmentation according to the size of a sliding window as model input, then processing the input data through a bidirectional LSTM layer formed by forward long-short term memory and backward long-short term memory to obtain coarse granularity characteristics of source data, then performing importance calculation on the previous coarse granularity characteristics by an attention mechanism layer to obtain fine granularity characteristics capable of reflecting activity characteristics, and finally processing the fine granularity characteristics by classified logistic regression to obtain probability distribution of a plurality of labels of the current data so as to finally judge the activity type. The invention improves the cognitive ability of the wearable sensor to the user activity, can accurately identify the user activity and improves the man-machine interaction ability.)

1. A human activity recognition method facing a wearable sensor is based on a layered deep learning model and is characterized in that: the method comprises the following steps:

(1) forming context fingerprint matrixes from time sequence heterogeneous data sensed by a wearable sensor, labeling the data by using a sliding window overlapping mechanism, and marking labels of activity types on the sensed data;

(2) processing input data through a bidirectional LSTM layer consisting of forward long-short term memory and backward long-short term memory to obtain coarse-grained characteristics of source data;

(3) calculating the importance of the coarse-grained features by using an Attention mechanism to obtain fine-grained features capable of reflecting activity characteristics;

(4) processing fine-grained features through a classified logistic regression method to obtain probability distribution of a plurality of labels of current data, wherein the maximum probability is the activity type of the current sensing data;

(5) and (3) training the network models in the steps (2) to (4) through the labeled data set in the step (1), and further obtaining a final layered deep learning model.

2. The wearable-sensor-oriented human activity recognition method of claim 1, wherein: the context fingerprint in the step (1) means that the human body behavior information perceived by the wearable sensor is integrated to be a context-invariant feature and can be used for subsequent data processing, and the context fingerprint matrix F ═ (F ═ F-₁,f₂,…,f_n) For the expression of chronologically heterogeneous data, wherein_i＝(Accx_i,Accy_i,Accz_i,Gyrx_i,Gyry_i,Gyrz_i,Magx_i,Magy_i,Magz_i,Com_i…..)^T，f_iThe elements in the data acquisition method are values of various wearable sensors, and i is a data acquisition point.

3. The wearable-sensor-oriented human activity recognition method of claim 1, wherein: in the step (1), data is segmented through a sliding window, the redundancy characteristic of the data is increased by using a window overlapping mechanism, and the data is labeled by using the activity type of the last data frame of each window.

4. The wearable-sensor-oriented human activity recognition method of claim 3, wherein: the sliding window size is set to 1500 ms.

5. The wearable-sensor-oriented human activity recognition method of claim 1, wherein: step (2) obtaining a hidden state h ═ h through a bidirectional LSTM model₀,h₁,…,h_t) Namely the extracted data coarse-grained characteristics,

6. The wearable-sensor-oriented human activity recognition method of claim 1, wherein: and (3) acquiring fine-grained features with activity distinguishing characteristics by using an Attention mechanism, namely learning the weights of the coarse-grained features extracted in the step (2) through the Attention mechanism, so as to obtain fine-grained features with feature preference, and reflecting unique characteristics presented when activities change.

7. The wearable-sensor-oriented human activity recognition method of claim 5, wherein: in step (3), firstly, the coarse-grained feature h is subjected to a nonlinear transformation to obtain an implicit expression value u, and the process can be represented as follows:

u＝tanh(W_u·h+b_u)，

on the basis of implicit expression, a normalized weight coefficient vector alpha capable of reflecting the importance of each element in u is learned through an Attention mechanism, so that the more the weight of the feature capable of reflecting the activity characteristic in the coarse-grained feature is obtained, and the finer-grained feature is obtained. The calculation expression of the weight coefficient α is:

thus, the fine-grained feature s can be expressed as:

8. the wearable-sensor-oriented human activity recognition method of claim 7, wherein: in step (4), the activity type result is calculated as:

y＝softmax(w_ls+b_l)。

9. the wearable-sensor-oriented human activity recognition method of claim 1, wherein: in the model training, a cross entropy loss function is used for evaluation, and when the cross entropy loss function tends to be converged in the training process, an optimal model is obtained.

Technical Field

The invention relates to the field of intelligent perception, mobile computing and pattern recognition, in particular to a human activity recognition method facing a wearable sensor.

Background

Human activity recognition refers to sensing human behavior data through various sensors, and then utilizing a computer automatic detection technology to analyze and understand various motion and behavior processes of a human body, wherein the technology has wide application scenes such as intelligent monitoring, human-computer interaction, robots and the like. In recent years, with the popularization of wearable devices with various built-in sensors, contact type human activity recognition based on the wearable sensors can be directly closely related to daily life of people, such as medical health monitoring or fitness monitoring. Therefore, activity recognition for wearable sensors has become a research focus in recent years.

Generally, wearable sensors are multi-channel, so that data sensed by the wearable sensors have characteristics of heterogeneity and time sequence, and can reflect characteristics of multi-dimensional movement of people, and therefore, human activity recognition facing the wearable sensors is generally considered as a classification problem of heterogeneous time sequence data. To solve this problem, some early scholars proposed an identification method based on data fusion, that is, a comprehensive characteristic is obtained by analyzing physical characteristics of multi-channel sensing data and then fusing multi-source data by methods such as weighted average, for example, a comprehensive acceleration value can be obtained by fusing triaxial acceleration information. And finally classifying the fused information by methods such as a Support Vector Machine (SVM), a Random Forest (RF), a Hidden Markov Model (HMM) and the like. However, this type of method belongs to an artificial feature extraction method, which is difficult to apply in a complex real-world environment because different people may have great differences for the same activity. In addition, the method neither reflects the characteristic of time continuity of the data, nor extracts internal features among heterogeneous data.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a wearable sensor-oriented human activity recognition method. The cognitive ability of the wearable sensor to the activities of the user is improved, the wearable sensor can be used as an auxiliary skill for reality enhancement, and the user experience is improved.

The invention is realized by the following technical scheme:

a wearable-sensor-oriented human activity recognition method, the method comprising the steps of:

(1) forming context fingerprint matrixes from time sequence heterogeneous data sensed by a wearable sensor, labeling the data by using a sliding window overlapping mechanism, and marking labels of activity types on the sensed data;

(2) processing input data through a bidirectional LSTM layer consisting of forward long-short term memory and backward long-short term memory to obtain coarse-grained characteristics of source data;

(3) calculating the importance of the coarse-grained features by using an Attention mechanism to obtain fine-grained features capable of reflecting activity characteristics;

(4) processing fine-grained features through a classified logistic regression method to obtain probability distribution of a plurality of labels of current data, wherein the maximum probability is the activity type of the current sensing data;

(5) and (3) training the network models in the steps (2) to (4) through the labeled data set in the step (1), and further obtaining a final layered deep learning model.

Further, the context fingerprint in step (1) refers to human behavior information perceived by integrating wearable sensors to make the human behavior information become context-invariant features and can be used for subsequent data processing, and the context fingerprint matrix F ═ F₁,f₂,…,f_n) For the expression of chronologically heterogeneous data, wherein_i＝(Accx_i,Accy_i,Accz_i,Gyrx_i,Gyry_i,Gyrz_i,Magx_i,Magy_i,Magz_i,Com_i…..)^T，f_iThe elements in the data acquisition method are values of various wearable sensors, and i is a data acquisition point.

Further, in the step (1), the data is segmented through a sliding window, the redundancy characteristic of the data is increased by utilizing a window overlapping mechanism, and the data is labeled by using the activity type of the last data frame of each window; further, the sliding window size is set to 1500ms optimal.

Further, the hidden state h obtained by the bidirectional LSTM model in step (2) is (h)₀,h₁,…,h_t) Namely the extracted data coarse-grained characteristics,

wherein the content of the first and second substances,

andare coarse-grained features on the data extracted by the forward LSTM and backward LSTM models, respectively.

Further, the step (3) of obtaining fine-grained features with activity discrimination characteristics by using an Attention mechanism means that the weights of the coarse-grained features extracted in the step (2) are learned through the Attention mechanism, so that the fine-grained features with feature preference are obtained, and unique characteristics presented when activities change can be reflected.

Further, in step (3), firstly, obtaining an implicit expression value u of the coarse-grained feature h through a nonlinear transformation, which can be expressed as:

u＝tanh(W_u·h+b_u)，

on the basis of implicit expression, a normalized weight coefficient vector alpha capable of reflecting the importance of each element in u is learned through an Attention mechanism, so that the more the weight of the feature capable of reflecting the activity characteristic in the coarse-grained feature is obtained, and the finer-grained feature is obtained. The calculation expression of the weight coefficient α is:

thus, the fine-grained feature s can be expressed as:

further, in step (4), the activity type result is calculated as:

y＝softmax(w_ls+b_l)。

further, in the step (5), in the model training, a cross entropy loss function is used for evaluation, and when the cross entropy loss function tends to converge in the training process, an optimal model is obtained.

The invention has the advantages and beneficial effects that:

(1) according to the invention, time series heterogeneous data sensed by a sensor is used as original data, and in the aspect of activity feature expression, fine-grained feature extraction with stronger distinctiveness is emphasized through a layered deep learning model, so that the feature can better reflect unique features presented when activities change, and the accuracy of activity identification can be improved. For this purpose, a context fingerprint matrix is firstly constructed as the input of the model; secondly, extracting coarse-grained characteristics of the original data by using a bidirectional LSTM model; then obtaining fine-grained characteristic expression of the original data according to an Attention mechanism; and finally, obtaining an activity recognition result through the multiple classifiers. The user activities can be accurately identified, and the man-machine interaction capability is improved.

(2) The cognitive ability of the wearable sensor to the activities of the user is improved, the wearable sensor can be used as an auxiliary skill for reality enhancement, and the user experience is improved.

(3) The activity recognition method provided by the invention has stronger robustness to the real environment, namely, under the complex environment, the model also has higher recognition precision and stable recognition speed, and has stronger transportability.

Drawings

FIG. 1 is a diagram of a layered deep learning model architecture for a wearable sensor oriented human activity recognition method of the present invention;

FIG. 2 is a diagram of a bi-directional LSTM model architecture;

FIG. 3 is a schematic diagram of data annotation based on a sliding window overlapping mechanism;

FIG. 4 is a diagram illustrating the classification results of activities under different sliding windows based on the OPPORTUNITY data set hierarchical deep learning model.

For a person skilled in the art, other relevant figures can be obtained from the above figures without inventive effort.

Detailed Description

In order to make the technical solution of the present invention better understood, the technical solution of the present invention is further described below with reference to specific examples.