阅读说明：本技术 基于多传感器融合的游泳监测方法、设备、存储介质和程序产品 (Swimming monitoring method, device, storage medium and program product based on multi-sensor fusion ) 是由 陈合意 吴保盛 文立勇 陈兴平 于 2021-06-28 设计创作，主要内容包括：本发明涉及到运动健康领域,具体提出了一种融合压力传感器(如气压计),磁力计,加速度计和陀螺仪等传感器的游泳监测方法,基于集成了气压计,磁力计,加速度计和陀螺仪的手部或腿部智能穿戴设备,如智能手表,智能手环或智能指环等,实现了入水检测,Stroke(划水),Lap(趟),Style(泳姿),Activity Time(划水时长),Rest Time(休息时长),Course(游程),Pace(游泳配速),SWOLF(游泳效率),Stroke Rate(划水频率)和StrokeLength(划水长度)等游泳数据的识别和计算,为游泳爱好者或康复训练者提供丰富和精确的游泳数据,支持用户更好地执行训练计划。(The invention relates to the field of sports health, and particularly provides a swimming monitoring method integrating pressure sensors (such as a barometer), a magnetometer, an accelerometer, a gyroscope and other sensors, which realizes the detection of water entry, the identification and calculation of swimming data such as Stroke, Lap, Style, Activity Time, Rest Time, run, Pace, SWOLF, Stroke Rate and Stroke length on the basis of hand or leg intelligent wearable equipment such as a smart watch, a smart bracelet or a smart ring integrating the barometer, the magnetometer, the accelerometer and the gyroscope, and provides rich and accurate swimming data for swimming enthusiasts or rehabilitation trainers to support the users to better execute training plans.)

1. The swimming monitoring method based on multi-sensor fusion is characterized by comprising the following steps of:

calculating a numerical value, receiving an original data packet, performing numerical value calculation, and outputting a physical quantity data packet, wherein the original data packet is formed by collecting data of multiple sensors by intelligent equipment, performing analog-to-digital conversion and then packaging;

the data cache is used for subtracting the standard atmospheric pressure value from the atmospheric pressure value output by the pressure sensor, keeping other data unchanged and caching the data into a data cache region;

pattern extraction, namely extracting a stroke cycle pattern based on the air pressure difference value cached in the data cache region;

calculating characteristics, namely calculating related characteristic values based on the stroke cycle data segment;

effective paddling identification, namely identifying whether the current paddling is effective paddling for one time or not based on the paddling characteristic set;

identifying the swimming stroke, namely identifying the swimming stroke value corresponding to the current stroke through the swimming stroke characteristic set on the premise that the current stroke is effective;

the branch trip identification is to identify whether the current trip is finished or not through a branch trip characteristic set on the premise that the current paddling period is effective, and if the current trip is finished, a complete branch trip is generated;

calculating branch data, namely calculating the completed branch data after the current branch is finished;

learning data, namely learning all lap data of the swimming after the swimming activity is finished;

data calibration, namely calibrating all lap data of the swimming based on the learned lap unit data, and deleting invalid lap and invalid stroke periods;

calculating statistical data, namely calculating the statistical data of the swimming after the swimming is finished;

and outputting data, and sending out the statistical data and the branch list data.

2. The multi-sensor fusion based swimming monitoring method according to claim 1, wherein: the sensors comprise a pressure sensor, an accelerometer, a gyroscope and a magnetometer; the physical quantity data packet comprises physical quantities of a pressure sensor, an accelerometer, a gyroscope and a magnetometer, wherein the physical quantities of the pressure sensor are air pressure and altitude, the physical quantities of the accelerometer are acceleration, the physical quantities of the gyroscope are angular velocity, and the physical quantities of the magnetometer are magnetic induction intensity;

signal processing is further included between the numerical value calculation step and the data caching step, and the physical quantity data packet is subjected to filtering processing through a filter;

in the pattern extraction step, a stroke cycle pattern is extracted according to physical quantity data of the pressure sensor, and the stroke cycle pattern is defined as a ternary vector of < front water outlet point, water inlet point and rear water outlet point >.

3. The method for swimming monitoring based on multi-sensor fusion according to claim 2, wherein: in the characteristic calculation step, calculating a lap division characteristic set based on the magnetic induction intensity data;

calculating a paddling feature set based on the air pressure difference value data, the accelerometer data and the gyroscope data, wherein the paddling feature set comprises cycle span, paddling force, water inlet duration, paddling depth and water outlet duration;

the calculation formula of the period Span is as follows:

Span＝EndTimeOfStroke-StartTimeOfStroke

wherein EndTimeOfStroke is the end time of the paddling, and StartTimeOfStroke is the start time of the paddling;

the stroke strength Stroke Strength calculation formula is as follows:

wherein, AccX, AccY, AccZ and AccM respectively represent X axis, Y axis, Z axis and modulus of the three-axis accelerometer, DtIndex_EntryIs the data index value DtIndex at the time of water entry of the paddling_{PstOutsurface}The data index value of the water outlet time of the water stroke is referred to;

the calculation formula of the water inlet time InwaterDur is as follows:

InwaterDur＝TimeOfPstOutsurface-TimeOfEntry

wherein, TimeOfPstOutsurface refers to the water outlet time of the paddling, and TimeOfEntry refers to the water inlet time of the paddling;

the calculation formula of the stroke depth InwaterDepth is as follows:

InwaterDepth＝Altitude_{PstOutsurface}-min(Altitude_Entry...Altitude_{PstOutsurface})

among them, Altitude_EntryIs the Altitude at the time of entering water of the paddling, namely Altitude_{PstOutsurface}The elevation at the water outlet time of the paddling is the altitude;

the calculation formula of the water outlet time OutwaterDur is as follows:

OutwaterDur＝TimeOfEntry-TimeOfPreOutSurface

wherein, TimeOfEntry refers to the water inlet time of the water stroke at this time, and TimeOfPreOutSurface refers to the water outlet time of the water stroke at the last time.

4. A multi-sensor fusion based swimming monitoring method according to claim 3, wherein: in the effective paddling identification step, judging whether the periodic span is smaller than a minimum span threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the water entry time length is larger than a maximum water entry time length threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the water entry time length is smaller than a minimum water entry time length threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the paddling depth is smaller than a minimum paddling depth threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the paddling force is smaller than a minimum paddling force threshold value, if so, skipping to the swimming gesture identification step.

5. A multi-sensor fusion based swimming monitoring method according to claim 3, wherein: in the step of swimming stroke identification, querying a swimming stroke table, judging whether the record is hit in the table lookup or not, if not, identifying that the swimming stroke value corresponding to the current stroke is 0, and skipping to the step of branch stroke identification, and if so, identifying that the swimming stroke value corresponding to the current stroke is 1-15, and skipping to the step of branch stroke identification; the swimming stroke value corresponding to breaststroke is 1, the swimming stroke value corresponding to free swimming is 2, the swimming stroke value corresponding to backstroke is 4, the swimming stroke value corresponding to butterfly swimming is 8, and the swimming stroke value corresponding to mixed swimming is 0.

6. The multi-sensor fusion based swimming monitoring method according to claim 5, wherein: in the step of branch pass identification, judging whether the difference value of the mean value of the magnetic induction intensity of the current stroke motion and the mean value of the mean values of the magnetic induction intensities of all the stroke motions in the current pass is larger than a set threshold value, if so, judging that the current pass is ended, otherwise, judging whether the vector of the magnetic induction intensity parameters of three branch shafts of the current stroke motion is equal to the vector of the magnetic induction intensity parameters of the existing stroke motion in the current pass, if so, judging that the current pass is ended, otherwise, skipping to the step of mode extraction;

in the step of calculating branch trip data, calculating the completed branch trip data, wherein the calculation of the completed branch trip data comprises a branch trip index, a branch trip distance, a paddling frequency, a main swimming stroke, a paddling duration, a speed distribution, a swimming efficiency, a paddling frequency and a paddling length;

the unit of the stroke duration is second, and the calculation formula is as follows:

wherein N represents the number of the stroke cycle, Span_iIs the span of the ith stroke cycle in the pass;

the pace is defined as the length of time spent per 100 meters;

the swimming efficiency is defined as the sum of the length of time used per 50 meters and the number of strokes;

the stroke frequency is defined as the number of strokes per minute;

the stroke length is defined as the distance of each stroke advancing;

the main swimming stroke identification process comprises the steps of reading the stroke movement of the current stroke, judging the swimming stroke of the current stroke movement, adding one to a stroke counter if the stroke movement is breaststroke, adding one to a free stroke counter if the stroke movement is free stroke, adding one to other swimming stroke counters if the stroke movement is mixed stroke, adding one to a backstroke counter if the stroke movement is backstroke, and adding one to a butterfly stroke counter if the stroke movement is butterfly stroke; judging whether all the stroke actions of the current trip are traversed or not, otherwise, skipping to reading the stroke actions of the current trip, if so, comparing counters, finding out that the swimming stroke corresponding to the maximum counter is the main swimming stroke, if the breaststroke counter is the maximum, the main swimming stroke is breaststroke, if the freestyle swimming counter is the maximum, the main swimming stroke is freestyle, if the backstroke counter is the maximum, the main swimming stroke is backstroke, if the butterfly stroke counter is the maximum, the main swimming stroke is butterfly stroke, and if other stroke counters are the maximum, the main swimming stroke is mixed stroke; and deleting all the stroke actions of the current stroke in the stroke action list.

7. A multi-sensor fusion based swimming monitoring method according to claim 3, wherein: in the data learning step, learning statistical parameters taking laps as units in the swimming;

in the data calibration step, calibrating all lap data of the swimming based on the learned statistical parameters taking laps as units;

in the step of calculating the statistical data, the swimming is subjected to statistical data calculation, wherein the statistical data calculation comprises total running length, total lap number, total stroke frequency, main swimming stroke, total stroke length, total rest time length, average stroke frequency, average stroke length, average speed and average stroke efficiency;

the total run length corresponds to the accumulation of the distances of all effective trips in the swimming activity, and the calculation formula is as follows:

wherein LapDatstance_iThe distance of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity;

the total stroke number corresponds to the accumulation of the stroke numbers of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, StrokeCount_iIs the number of times of stroke in the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the total stroke duration corresponds to the accumulation of the stroke durations of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, ActTime_iThe stroke duration of the ith effective stroke in the swimming activity is shown, and the LapCount is the total effective stroke number in the swimming activity;

the average stroke frequency corresponds to the cumulative sum of the stroke frequencies of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, StrokeRate_iIs the stroke frequency of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average stroke length corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, Stroke Length_iIs the stroke length of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average speed corresponds to the cumulative sum of the stroke frequency of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, Pace_iIs the matching speed of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity;

the average stroke efficiency corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity divided by the total stroke number, and the calculation formula is as follows:

wherein, SWOLF_iThe stroke efficiency of the ith effective stroke in the swimming activity is shown, and the LapCount is the total effective stroke number in the swimming activity;

the main swimming stroke calculation process is to traverse all strokes of the water stroke, judge the current swimming stroke, if the stroke is breaststroke, the breaststroke counter is incremented, if the stroke is free stroke, the free stroke counter is incremented, if the stroke is mixed stroke, the other swimming stroke counters are incremented, if the stroke is backstroke, the backstroke counter is incremented, and if the stroke is butterfly stroke, the butterfly stroke counter is incremented; and judging whether all the strokes traverse, otherwise, jumping to all the strokes traversing the current stroke, if so, comparing the counters, finding out that the swimming posture corresponding to the maximum counter is the main swimming posture of the current stroke, if the breaststroke counter is the maximum, the main swimming posture of the current stroke is breaststroke, if the freestyle stroke counter is the maximum, the main swimming posture of the current stroke is freestyle stroke, if the backstroke counter is the maximum, the main swimming posture of the current stroke is backstroke, if the butterfly stroke counter is the maximum, the main swimming posture of the current stroke is butterfly stroke, and if other stroke counters are the maximum, the main swimming posture of the current stroke is mixed stroke.

8. An electronic device, characterized by comprising: a processor;

a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for carrying out the method according to any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program is executed by a processor for performing the method according to any of claims 1-7.

10. A computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the method according to any of claims 1-7.

Technical Field

The invention relates to the technical field of sports health, in particular to a swimming monitoring method, equipment, a storage medium and a program product based on multi-sensor fusion.

Background

Swimming sports, as a traditional sports item, can be used for fat reduction and shaping, improving the functions of a cardiovascular system, improving the functions of a respiratory system, improving the coordination ability of the body, improving the body temperature regulation ability, improving the immunity ability of the body, improving the survival rate of people in danger in water and the like, and is always highly popular and loved by various people. As the swimming sport is popular with people, large-scale competitive sports events such as world swimming championship games, world cup games, world short pool championship games and the like are promoted. In addition, hydrotherapy is considered as a main treatment method in physiotherapy, so that swimming movements related to water tightness are also applied to promote rehabilitation of patients such as spine diseases, knee joints, ankle joints and the like in cooperation with clinical treatment.

The traditional swimming monitoring method is mainly completed through a camera and a video analysis technology, and the method has the defects of high cost, troublesome installation, poor usability, weak anti-interference capability, easy privacy exposure and the like, and is difficult to popularize and apply generally.

With the development of microcomputer systems and sensing technologies, smart devices become more and more popular and miniaturized, and more companies and research institutions begin to research and apply human behavior recognition based on various sensing technologies built in smart devices. At present, be applied to swimming motion's smart machine, like intelligent wrist-watch, intelligent bracelet, intelligent swimming goggles, products such as intelligent rhythm of the heart ware and intelligent foot ring begin to appear on the market, like HUAWEI GT series smart watch, swift Poolmate series smart watch, the intelligent wrist-watch of Garmin swimming series, Misfit intelligent bracelet, sunto intelligent wrist-watch, Instabeat intelligent swimming goggles, finish's ear clip formula rhythm of the heart ware and Flyfit intelligent foot ring etc.. Various intelligent devices applied to swimming sports have different sensor technologies, different positions where people wear the intelligent devices, different swimming related data and different data accuracy. For example, a smart watch and a smart bracelet basically adopt motion sensors such as an accelerometer or a gyroscope, and can output data such as Strokes, Lap and swim gestures, but because the motion sensors are very sensitive to motions, even if the motions are Swimming in the same Swimming gesture, the fine motions of the hands of each person are very different, so that the coverage of people is limited and the algorithm model is very complex. On the other hand, since the motion sensor can only sense the motion, the stroke motion in the swimming state and the similar motion in the non-swimming state are difficult to accurately distinguish, the data is easy to be interfered to reduce the accuracy, for example, the non-swimming motion of the hand during rest in the swimming process can be mistakenly judged as the swimming motion to generate the swimming data. The intelligent foot ring adopts an accelerometer, can output data such as the number of times of pedaling, the frequency of pedaling, the run length and the like, and has single data. The intelligence swimming mirror passes through photoelectric sensor and obtains the rhythm of the heart from temporal artery, and data is comparatively single, and receives the influence of water impact force easily and reduces the data accuracy. Therefore, a swimming monitoring method which provides richer and more accurate swimming related data by fusing a plurality of sensing technologies and provides reference and guidance for swimming competition training or swimming rehabilitation training is urgently needed.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention aims to provide a swimming monitoring method based on multi-sensor fusion, which can sense the water entry state, the swimming action, the swimming direction and the swimming state by fusing a plurality of sensors such as a pressure sensor (such as a barometer), an accelerometer, a gyroscope, a magnetometer and the like, can judge whether the swimming is really underwater swimming or not, improves the anti-interference performance of the swimming process and provides richer and more accurate swimming data.

The invention provides a swimming monitoring method based on multi-sensor fusion, which comprises the following steps:

calculating a numerical value, receiving an original data packet, performing numerical value calculation, and outputting a physical quantity data packet, wherein the original data packet is formed by collecting data of multiple sensors by intelligent equipment, performing analog-to-digital conversion and then packaging;

the data cache is used for subtracting the standard atmospheric pressure value from the atmospheric pressure value output by the pressure sensor, keeping other data unchanged and caching the data into a data cache region;

pattern extraction, namely extracting a stroke cycle pattern based on the air pressure difference value cached in the data cache region;

calculating characteristics, namely calculating related characteristic values based on the stroke cycle data segment;

effective paddling identification, namely identifying whether the current paddling is effective paddling for one time or not based on the paddling characteristic set;

identifying the swimming stroke, namely identifying the swimming stroke value corresponding to the current stroke through the swimming stroke characteristic set on the premise that the current stroke is effective;

the branch trip identification is to identify whether the current trip is finished or not through a branch trip characteristic set on the premise that the current paddling period is effective, and if the current trip is finished, a complete branch trip is generated;

calculating branch data, namely calculating the completed branch data after the current branch is finished;

learning data, namely learning all lap data of the swimming after the swimming activity is finished;

data calibration, namely calibrating all lap data of the swimming based on the learned lap unit data, and deleting invalid lap and invalid stroke periods;

calculating statistical data, namely calculating the statistical data of the swimming after the swimming is finished;

and outputting data, and sending out the statistical data and the branch list data.

Further, the sensors include pressure sensors, accelerometers, gyroscopes, and magnetometers; the physical quantity data packet comprises physical quantities of a pressure sensor, an accelerometer, a gyroscope and a magnetometer, wherein the physical quantities of the pressure sensor are air pressure and altitude, the physical quantities of the accelerometer are acceleration, the physical quantities of the gyroscope are angular velocity, and the physical quantities of the magnetometer are magnetic induction intensity;

signal processing is further included between the numerical value calculation step and the data caching step, and the physical quantity data packet is subjected to filtering processing through a filter;

in the pattern extraction step, a stroke cycle pattern is extracted according to physical quantity data of the pressure sensor, and the stroke cycle pattern is defined as a ternary vector of < front water outlet point, water inlet point and rear water outlet point >.

Further, in the feature calculation step, a lap feature set is calculated based on the magnetic induction intensity data;

calculating a stroke feature set including a cycle span, a stroke strength, a water entry duration, a stroke depth and a water exit duration based on the air pressure difference value data, the accelerometer data and the gyroscope data;

the calculation formula of the period Span is as follows:

Span＝EndTimeOfStroke-StartTimeOfStroke

wherein EndTimeOfStroke is the end time of the paddling, and StartTimeOfStroke is the start time of the paddling;

the stroke strength Stroke Strength calculation formula is as follows:

wherein, AccX, AccY, AccZ and AccM respectively represent X axis, Y axis, Z axis and modulus of the three-axis accelerometer, DtIndex_EntryIs the data index value DtIndex at the time of water entry of the paddling_{PstOutsurface}The water outlet time is a data index value of the water outlet time of the water stroke;

the calculation formula of the water inlet time InwaterDur is as follows:

InwaterDur＝TimeOfPstOutsurface-TimeOfEntry

wherein, TimeOfPstOutsurface refers to the water outlet time of the paddling, and TimeOfEntry refers to the water inlet time of the paddling;

the calculation formula of the stroke depth InwaterDepth is as follows:

InwaterDepth＝Altitude_{PstOutsurface}-min(Altitude_Entry...Altitude_{PstOutsurface})

among them, Altitude_EntryIs the Altitude at the time of entering water of the paddling, namely Altitude_{PstOutsurface}The altitude of the water outlet moment of the paddling is the altitude;

the calculation formula of the water outlet time OutwaterDur is as follows:

OutwaterDu r＝TimeOfEntr y-TimeOf Pr eOutSurfac e

wherein, TimeOfEntry refers to the water inlet time of the water stroke at this time, and TimeOfPreOutSurface refers to the water outlet time of the water stroke at the last time.

Further, in the effective paddling identification step, judging whether the cycle span is smaller than a minimum span threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the entry duration is larger than a maximum entry duration threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the entry duration is smaller than a minimum entry duration threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the paddling depth is smaller than a minimum paddling depth threshold value, if so, skipping to the mode extraction step, otherwise, judging whether the paddling force is smaller than a minimum paddling force threshold value, if so, skipping to the mode extraction step, and otherwise, skipping to the swimming posture identification step.

Further, in the step of swimming stroke identification, querying a swimming stroke table, judging whether the record is hit in the table lookup, if not, identifying that the swimming stroke value corresponding to the current stroke is 0, and skipping to the step of branch stroke identification, and if so, identifying that the swimming stroke value corresponding to the current stroke is 1-15, and skipping to the step of branch stroke identification; the swimming stroke value corresponding to breaststroke is 1, the swimming stroke value corresponding to free swimming is 2, the swimming stroke value corresponding to backstroke is 4, the swimming stroke value corresponding to butterfly swimming is 8, and the swimming stroke value corresponding to mixed swimming is 0.

Further, in the step of branch pass identification, judging whether the difference value between the mean value of the magnetic induction intensity of the current stroke action and the mean value of the mean values of the magnetic induction intensity of all the stroke actions in the current pass is greater than a set threshold value, if so, judging that the current pass is ended, otherwise, judging whether the magnetic induction intensity parameter vectors of three branch axes of the current stroke action are equal to the magnetic induction intensity parameter vectors of the existing stroke actions in the current pass, if so, judging that the current pass is ended, otherwise, skipping to the step of mode extraction;

in the step of calculating branch trip data, calculating the completed branch trip data, wherein the calculation of the completed branch trip data comprises a branch trip index, a branch trip distance, a water stroke frequency, a main swimming stroke, a water stroke duration, a speed distribution, a swimming efficiency, a water stroke frequency and a water stroke length;

the unit of the stroke duration is second, and the calculation formula is as follows:

wherein N represents the number of the stroke cycle, Span_iIs the span of the ith stroke cycle in the pass;

the pace is defined as the length of time spent per 100 meters;

the swimming efficiency is defined as the sum of the length of time used per 50 meters and the number of strokes;

the stroke frequency is defined as the number of strokes per minute;

the stroke length is defined as the distance of each stroke advancing;

the main swimming stroke identification process comprises the steps of reading the stroke movement of the current stroke, judging the swimming stroke of the current stroke movement, adding one to a stroke counter if the stroke movement is breaststroke, adding one to a free stroke counter if the stroke movement is free stroke, adding one to other swimming stroke counters if the stroke movement is mixed stroke, adding one to a backstroke counter if the stroke movement is backstroke, and adding one to a butterfly stroke counter if the stroke movement is butterfly stroke; judging whether all stroke motions of the current stroke are traversed or not, otherwise, skipping to reading the stroke motions of the current stroke, if so, comparing counters, finding out that the swimming stroke corresponding to the maximum counter is the main swimming stroke, if the breaststroke counter is the maximum, the main swimming stroke is breaststroke, if the freestyle swimming counter is the maximum, the main swimming stroke is freestyle, if the backstroke counter is the maximum, the main swimming stroke is backstroke, if the butterfly stroke counter is the maximum, the main swimming stroke is butterfly stroke, and if the other stroke counters are the maximum, the main swimming stroke is mixed stroke; and deleting all the stroke actions of the current stroke in the stroke action list.

Further, in the data learning step, learning statistical parameters taking laps as units in the swimming;

in the data calibration step, calibrating all lap data of the swimming based on the learned statistical parameters taking laps as units;

in the step of calculating the statistical data, the swimming is subjected to statistical data calculation, wherein the statistical data calculation comprises total running length, total lap number, total stroke frequency, main swimming stroke, total stroke length, total rest time length, average stroke frequency, average stroke length, average speed and average stroke efficiency;

the total run length corresponds to the accumulation of the distances of all effective trips in the swimming activity, and the calculation formula is as follows:

wherein LapDatstance_iThe distance of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity;

the total stroke number corresponds to the accumulation of the stroke numbers of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, StrokeCount_iIs the number of times of the ith effective lap in the swimming activity, and the LapCount is the total number of the effective laps in the swimming activity;

the total stroke duration corresponds to the accumulation of the stroke durations of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, ActTime_iIs that the ith in the swimming activity isThe effective stroke length is the total effective stroke number in the swimming activity;

the average stroke frequency corresponds to the cumulative sum of the stroke frequencies of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, StrokeRate_iIs the stroke frequency of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average stroke length corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, Stroke Length_iIs the stroke length of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average speed corresponds to the cumulative sum of the stroke frequency of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, Pace_iIs the matching speed of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity;

the average stroke efficiency corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity divided by the total stroke number, and the calculation formula is as follows:

wherein S isWOLF_iThe stroke efficiency of the ith effective stroke in the swimming activity is shown, and the LapCount is the total effective stroke number in the swimming activity;

the main swimming posture calculation process is to traverse all strokes of the water stroke, judge the current swimming posture, if the swimming posture is breaststroke, the breaststroke counter is increased by one, if the swimming posture is free swimming, the free swimming counter is increased by one, if the swimming posture is mixed swimming, the other swimming posture counters are increased by one, if the swimming posture is backstroke, the backstroke counter is increased by one, and if the swimming posture is butterfly, the butterfly stroke counter is increased by one; and judging whether all the strokes traverse, otherwise jumping to traverse all the strokes of the water stroke, if so, comparing the counters, finding out that the swimming posture corresponding to the maximum counter is the main swimming posture of the water stroke, if the breaststroke counter is the maximum, the main swimming posture of the water stroke is breaststroke, if the freestyle swimming counter is the maximum, the main swimming posture of the water stroke is freestyle, if the backstroke counter is the maximum, the main swimming posture of the water stroke is backstroke, if the butterfly stroke counter is the maximum, the main swimming posture of the water stroke is butterfly stroke, and if other swimming posture counters are the maximum, the main swimming posture of the water stroke is mixed stroke.

An electronic device, comprising: a processor;

a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a multi-sensor fusion based swim monitoring method.

A computer-readable storage medium, on which a computer program is stored, which is executed by a processor for a method for swim monitoring based on multi-sensor fusion.

A computer program product comprising a computer program/instructions which, when executed by a processor, implements a multi-sensor fusion based swimming monitoring method.

Compared with the prior art, the invention has the beneficial effects that:

the invention can sense the water entry state, the swimming action, the swimming direction and the swimming state by fusing a plurality of sensors such as a pressure sensor (such as a barometer), an accelerometer, a gyroscope, a magnetometer and the like, can judge whether the swimming is really underwater swimming or not, improves the anti-interference performance of the swimming process, provides richer and more accurate swimming data, provides reference and guide for swimming competitive training or swimming rehabilitation training and supports a user to better execute a training plan.

The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood and to be implemented according to the content of the specification, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings. The detailed description of the present invention is given in detail by the following examples and the accompanying drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:

FIG. 1 is a functional block diagram of a swimming monitoring system based on multi-sensor fusion according to an embodiment of the present invention;

FIG. 2 is a flow chart of a swimming monitoring method based on multi-sensor fusion according to the present invention;

FIG. 3 is a schematic view of a stroke cycle in accordance with an embodiment of the present invention;

FIG. 4 is a flow chart of an efficient stroke recognition according to an embodiment of the present invention;

FIG. 5 is a Stroke gesture recognition flowchart according to an embodiment of the present invention;

FIG. 6 is a flow chart of split pass identification according to an embodiment of the present invention;

FIG. 7 is a flow chart of main swimming stroke recognition in lap division according to the embodiment of the present invention;

fig. 8 is a flow chart of main swimming stroke recognition of the current swimming activity according to the embodiment of the present invention.

Detailed Description

The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.

At present, in the market, the intelligent wearable product for swimming basically only uses motion sensors such as an accelerometer or a gyroscope. The products only can identify Stroke actions (strokes), whether the current trip (Lap) is finished or not is estimated through the difference of the Stroke actions or the interval duration, and then the swimming postures (styles) are identified through the difference of the Stroke actions (strokes) under different swimming postures (styles).

In fact, different individuals and different swimming postures have different Stroke motions, and even in the same swimming posture, the Stroke motions (strokes) of each swimming enthusiast are different, and physical performance reduction and muscle fatigue during swimming inevitably have adverse effects on the stability of the Stroke motions (strokes). Therefore, only by the swimming monitoring method of recognizing the Stroke motion (Stroke) and the difference thereof, an error is generated by various adverse factors at the time of recognizing the Stroke motion (Stroke), and is superimposed on the Lap (Lap) and the recognized swimming Stroke (Style) divided by the difference or interval of the Stroke motion (Stroke), and it is difficult to obtain accurate swimming data.

The invention can sense the water entry state, the swimming action, the swimming direction and the swimming state by fusing a plurality of sensors such as a pressure sensor (such as a barometer), an accelerometer, a gyroscope, a magnetometer and the like, can judge whether the swimming is really underwater swimming or not, improves the anti-interference performance in the swimming process and provides richer and more accurate swimming data.

As shown in fig. 1, the system is connected to a plurality of sensors, including pressure sensors (e.g., barometers), accelerometers, gyroscopes, and magnetometers, a display unit, a control unit, a transmission unit, and a storage unit through a processing unit.

The swimming monitoring method based on multi-sensor fusion, as shown in fig. 2, comprises the following steps:

the intelligent device, such as a smart watch, a smart bracelet or a smart ring, collects sensor data such as a pressure sensor (such as a barometer), an accelerometer, a gyroscope, a magnetometer and the like at a sampling frequency of 25Hz or more, and packages the data into an original data packet after A/D conversion is completed.

After the numerical value calculation is finished, filtering the physical quantity data packet by designing a related filter, and filtering or attenuating noise in the data;

calculating a numerical value, receiving an original data packet, performing numerical calculation, and outputting a physical quantity data packet, wherein physical quantities of a barometer are air pressure and altitude, physical quantities of an accelerometer are acceleration, physical quantities of a gyroscope are angular velocity, and physical quantities of a magnetometer are magnetic induction intensity;

data buffer, after finishing signal processing, subtracting standard atmospheric pressure value (1.01325 × 10) from atmospheric pressure value output by pressure sensor⁵) Caching the Data into a Data Buffer area (Data Buffer) to be subjected to mode extraction when other Data are kept unchanged;

and pattern extraction, namely extracting the stroke cycle pattern based on the air pressure difference value cached in the data cache region. Therefore, the stroke cycle pattern is defined as < PreOutsurface (front water out point), Entry (water in point), PstOutsurface (rear water out point) > ternary vector, as shown in FIG. 3.

And (4) feature calculation, wherein after the paddling period mode extraction is completed, related feature values are calculated based on the paddling period data segments. Such as: calculating a Lap feature Set (Lap Ft Set) based on the magnetic induction data;

calculating a Stroke feature Set (Stroke Ft Set) based on air pressure difference value data, triaxial accelerometer data and triaxial gyroscope data, wherein the Stroke feature Set comprises parameters such as cycle Span (duration), Stroke strength, water entering duration, Stroke depth and water outlet duration, and a cycle Span (duration) Span calculation formula is as follows:

Span＝EndTimeOfStroke-StartTimeOfStroke

wherein EndTimeOfStroke is the end time of the paddling, and StartTimeOfStroke is the start time of the paddling;

the calculation formula of the paddling force Stroke Strength is as follows:

wherein, AccX, AccY, AccZ and AccM respectively represent X axis, Y axis, Z axis and modulus of the three-axis accelerometer, DtIndex_EntryIs the data index value DtIndex at the time of water entry of the paddling_{PstOutsurface}The water outlet time is a data index value of the water outlet time of the water stroke;

the water entry time InwaterDur calculation formula is as follows:

InwaterDur＝TimeOfPstOutsurface-TimeOfEntry

wherein, TimeOfPstOutsurface refers to the water outlet time of the paddling, and TimeOfEntry refers to the water inlet time of the paddling;

the calculation formula of the stroke depth InwaterDepth is as follows:

InwaterDepth＝Altitude_{PstOutsurface}-min(Altitude_Entry...Altitude_{PstOutsurface})

among them, Altitude_EntryIs the Altitude at the time of entering water of the paddling, namely Altitude_{PstOutsurface}The altitude of the water outlet moment of the paddling is the altitude;

the calculation formula of the water outlet time OutwaterDur is as follows:

OutwaterDu r＝TimeOfEntr y-TimeOf Pr eOutSurfac e

wherein, TimeOfEntry refers to the water inlet time of the water stroke at this time, and TimeOfPreOutSurface refers to the water outlet time of the water stroke at the last time.

Effective paddling identification, namely identifying whether the current paddling is effective paddling for one time or not based on a paddling characteristic Set (Stroke Ft Set), judging whether the cycle span is smaller than a minimum span threshold value or not as shown in fig. 4, if so, jumping to a mode extraction step, otherwise, judging whether the water entry time length is larger than a maximum water entry time length threshold value or not, if so, jumping to the mode extraction step, otherwise, judging whether the water entry time length is smaller than a minimum water entry time length threshold value or not, if not, judging whether the paddling depth is smaller than a minimum paddling depth threshold value or not, jumping to the mode extraction step, otherwise, jumping to the swimming gesture identification step.

And (2) swimming stroke identification, wherein under the premise that the current paddling is effective, a swimming stroke value corresponding to the current paddling is identified by a swimming stroke characteristic Set (Style Ft Set) by 0-15, and the identification process is shown in FIG. 5 by (Breast stroke Breaststroke: 1, Freestyle swimming: 2, backstick Backstroke: 4, Butterfly and Medley hybrid Butterfly: 0). The method specifically comprises the following steps: and inquiring the swimming stroke table, judging whether the record is hit in the table lookup, if not, identifying that the swimming stroke value corresponding to the current stroke is 0, and skipping to the step of branch stroke identification, and if so, identifying that the swimming stroke value corresponding to the current stroke is 1-15, and skipping to the step of branch stroke identification.

And (3) branch trip identification, wherein on the premise that the current stroke cycle is effective, whether the current trip is ended is identified through a branch trip feature Set (Lap Ft Set), if the current trip is ended, a complete branch trip is generated, and the identification process is shown in fig. 6. Specifically, whether the difference value between the mean value of the magnetic induction intensity of the current stroke motion and the mean value of the mean values of the magnetic induction intensity of all the stroke motions in the current stroke is larger than a set threshold value or not is judged, if yes, the current stroke is judged to be finished, otherwise, whether the vector of the magnetic induction intensity parameters of the three split axes of the current stroke motion is equal to the vector of the magnetic induction intensity parameters of the existing stroke motions in the current stroke or not is judged, if yes, the current stroke is judged to be finished, and if not, the mode extraction step is skipped to.

And (2) calculating branch data, after the current branch is finished, calculating the completed branch data, including data items such as a branch Index (Lap Index), a branch Distance (Lap Distance, generally a lane Length or a Distance obtained by other means such as a GPS), a Stroke number (Stroke Count), a Main swimming Stroke (Main Style), a Stroke duration (Activity Time), a Pace (Pace), a swimming efficiency (SWOLF), a Stroke frequency (Stroke Rate), a Stroke Length (Stroke Length) and the like, wherein the calculation formula of the related data items is as follows, and the identification process of the Main swimming Stroke (Main Style) is as shown in fig. 7.

The unit of the stroke length is second, and the calculation formula is as follows:

wherein N represents the number of the stroke cycle, Span_iIs the span of the ith stroke cycle in the pass;

pacing is defined as the length of time spent per 100 meters;

swimming efficiency is defined as the sum of the length of time used per 50 meters and the number of strokes;

stroke frequency is defined as the number of strokes per minute;

the stroke length is defined as the distance of each stroke advancing;

the main swimming stroke identification process comprises the steps of reading the stroke movement of the current stroke, judging the swimming stroke of the current stroke movement, adding one to a stroke counter if the stroke movement is breaststroke, adding one to a free stroke counter if the stroke movement is free stroke, adding one to other swimming stroke counters if the stroke movement is mixed stroke, adding one to a backstroke counter if the stroke movement is backstroke, and adding one to a butterfly stroke counter if the stroke movement is butterfly stroke; judging whether all the stroke actions of the current trip are traversed or not, otherwise, skipping to reading the stroke actions of the current trip, if so, comparing the counters, finding out that the stroke posture corresponding to the maximum counter is the main stroke, if the stroke counter is the maximum, the main stroke is breaststroke, if the freestyle stroke counter is the maximum, the main stroke is freestyle stroke, if the backstroke counter is the maximum, the main stroke is backstroke, if the backstroke counter is the maximum, the main stroke is butterfly stroke, and if other stroke counters are the maximum, the main stroke is mixed stroke; and deleting all the stroke actions of the current stroke in the stroke action list.

Learning data, namely learning all lap data of the swimming after the swimming activity is finished; and learning the statistical parameter of the Stroke Count (Stroke Count) in the unit of lap in the swimming.

And (4) calibrating data, namely calibrating all branch data of the swimming according to the statistical parameter of the Stroke Count (Stroke Count) which is learned by taking the Stroke as a unit after the data are learned, and deleting invalid strokes and invalid Stroke periods.

And calculating statistical data, wherein after the data calibration is completed, the statistical data is calculated for the current swimming, and the statistical data comprises total run (Count), total trip (Laps), total Stroke number (Stroke Count), Main swimming Stroke (Main Style), total Stroke Time (Activity Time), total Rest Time (Rest Time), average Stroke frequency (Stroke Rate), average Stroke Length (Stroke Length), average speed distribution (Pace) and average Stroke efficiency (SWOLF) and the like. The calculation process of the Main swimming stroke (Main Style) is shown in fig. 8.

The total run corresponds to the accumulation of the distances of all effective trips in the swimming activity, and the calculation formula is as follows:

wherein LapDatstance_iThe distance of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity;

the total stroke number corresponds to the accumulation of the stroke numbers of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, StrokeCount_iIs the number of times of the ith effective lap in the swimming activity, and the LapCount is the total number of the effective laps in the swimming activity;

the total stroke duration corresponds to the accumulation of the stroke durations of all effective strokes in the swimming activity, and the calculation formula is as follows:

wherein, ActTime_iThe stroke duration of the ith effective stroke in the swimming activity is shown, and the LapCount is the total effective stroke number in the swimming activity;

the average stroke frequency corresponds to the cumulative sum of the stroke frequencies of all effective strokes in the swimming activity and is divided by the total number of strokes, and the calculation formula is as follows:

wherein, StrokeRate_iIs the stroke frequency of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average stroke length corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity and is divided by the total stroke number, and the calculation formula is as follows:

wherein, Stroke Length_iIs the stroke length of the ith effective stroke in the swimming activity, and the LapCount is the total effective stroke number in the swimming activity;

the average speed corresponds to the cumulative sum of the stroke frequency of all effective strokes in the swimming activity and is divided by the total number of strokes, and the calculation formula is as follows:

wherein, Pace_iIs the matching speed of the ith effective lap in the swimming activity, and the LapCount is the total effective lap number in the swimming activity.

The average stroke efficiency corresponds to the stroke frequency accumulation sum of all effective strokes in the swimming activity divided by the total stroke number, and the calculation formula is as follows:

wherein, SWOLF_iThe stroke efficiency of the ith effective stroke in the swimming activity is shown, and the LapCount is the total effective stroke number in the swimming activity;

the main swimming stroke calculation process is to traverse all strokes of the water stroke, judge the current swimming stroke, if the stroke is breaststroke, the breaststroke counter is increased by one, if the stroke is free stroke, the free stroke counter is increased by one, if the stroke is mixed stroke, the other swimming stroke counters are increased by one, if the stroke is backstroke, the backstroke counter is increased by one, and if the stroke is butterfly stroke, the butterfly stroke counter is increased by one; and judging whether all the strokes traverse, otherwise, jumping to all the strokes of the current stroke, if so, comparing the counters, finding out that the swimming posture corresponding to the maximum counter is the main swimming posture of the current stroke, if the breaststroke counter is the maximum, the main swimming posture of the current stroke is breaststroke, if the freestyle stroke counter is the maximum, the main swimming posture of the current stroke is freestyle stroke, if the backstroke counter is the maximum, the main swimming posture of the current stroke is backstroke, if the butterfly stroke counter is the maximum, the main swimming posture of the current stroke is butterfly stroke, and if other stroke counters are the maximum, the main swimming posture of the current stroke is mixed stroke.

And outputting data, namely after the calculation of the statistical data is completed, sending the statistical data and the branch list data.

An electronic device, comprising: a processor;

a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a swimming monitoring method based on multi-sensor fusion.

A computer-readable storage medium, on which a computer program is stored, which is executed by a processor for a method for swim monitoring based on multi-sensor fusion.

A computer program product comprising a computer program/instructions which, when executed by a processor, implements a multi-sensor fusion based swimming monitoring method.

The invention relates to the field of sports health, and particularly provides a swimming monitoring method integrating pressure sensors (such as a barometer), a magnetometer, an accelerometer, a gyroscope and other sensors, which realizes the identification and calculation of swimming data such as Stroke, Lap, swimming Stroke, Activity Time, Rest Time, Corse, Pace, SWOLF (swimming efficiency), Stroke Rate and Stroke Length on the basis of hand or leg intelligent wearable equipment such as a smart watch, a smart bracelet or a smart ring and the like integrating the barometer, the magnetometer, the accelerometer and the gyroscope, thereby providing rich and accurate swimming data for swimming enthusiasts or rehabilitation trainees and supporting the users to better execute training plans.

The foregoing is merely a preferred embodiment of the invention and is not intended to limit the invention in any manner; those skilled in the art can readily practice the invention as shown and described in the drawings and detailed description herein; however, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the scope of the invention as defined by the appended claims; meanwhile, any changes, modifications, and evolutions of the equivalent changes of the above embodiments according to the actual techniques of the present invention are still within the protection scope of the technical solution of the present invention.