阅读说明：本技术 一种基于运动检测辅助识别的打击乐智能教育系统 (Percussion music intelligent education system based on motion detection auxiliary recognition ) 是由 高建平 于 2021-09-16 设计创作，主要内容包括：本发明涉及一种基于运动检测辅助识别的打击乐智能教育系统,包括鼓棒、架子鼓和主机,鼓棒上安装有加速度传感器和陀螺仪传感器,主机还用于采集架子鼓的音频数据,数据处理过程包括在静止时刻,获取重力加速度值,确定鼓棒所握的位置；鼓棒动作时,根据陀螺仪传感器采集旋转量,基于静止时的重力加速度值,计算当前的重力加速度,然后与当前加速度传感器获取的数据作差,得到鼓棒去除重力后的加速度,最后结合鼓棒头部位置,计算鼓棒的打击坐标力度；根据音频数据提取音频特征；根据音频特征和鼓棒的打击坐标力度通过神经网络识别音色和力度,用于评分。与现有技术相比,本发明具有识别率高、打击时间获取准确、增加了手法识别等优点。(The invention relates to a percussion music intelligent education system based on motion detection auxiliary recognition, which comprises a drum stick, a drum kit and a host, wherein the drum stick is provided with an acceleration sensor and a gyroscope sensor, the host is also used for collecting audio data of the drum kit, and the data processing process comprises the steps of acquiring a gravity acceleration value at a static moment and determining the holding position of the drum stick; when the drum stick acts, acquiring rotation amount according to a gyroscope sensor, calculating current gravity acceleration based on a gravity acceleration value during static, then making a difference with data acquired by a current acceleration sensor to obtain acceleration of the drum stick after gravity removal, and finally calculating the striking coordinate strength of the drum stick by combining the head position of the drum stick; extracting audio features according to the audio data; and identifying tone and strength through a neural network according to the audio features and the striking coordinate strength of the drumstick for grading. Compared with the prior art, the method has the advantages of high recognition rate, accurate striking time acquisition, increased manipulation recognition and the like.)

1. The utility model provides a percussion music intelligence educational system based on supplementary discernment of motion detection, includes drum stick, shelf drum and host computer, its characterized in that, install acceleration sensor and gyroscope sensor on the drum stick, acceleration sensor and the equal communication connection of gyroscope sensor the host computer, the host computer still is used for gathering the audio data of shelf drum, the data processing process of host computer includes following step:

an initial correction step: at a static moment before the music begins, obtaining a gravity acceleration value according to a detection value of the acceleration sensor, and determining the current position held by the drum stick;

identifying the strength of the striking coordinate: when the drum stick acts, acquiring the rotation amount of the drum stick according to the gyroscope sensor, calculating the current gravitational acceleration according to the gravitational acceleration value acquired in the initial correction step, then subtracting the current gravitational acceleration from the data acquired by the current acceleration sensor to obtain the acceleration of the drum stick after gravity is removed, acquiring the position of the head of the drum stick according to the position held by the drum stick, and finally calculating the striking coordinate strength of the drum stick according to the momentum theorem;

audio characteristic extraction: extracting audio features according to the audio data;

identifying percussion music: loading the audio features and the striking coordinate strength of the drumstick into a pre-established and trained striking recognition neural network model to obtain the tone and strength of the percussion instrument,

grading: and giving a music score according to the tone and the strength of the percussion instrument.

2. The percussion music intelligent education system based on motion detection and auxiliary recognition according to claim 1, characterized in that the number of the acceleration sensors is three, and the three acceleration sensors are arranged in a pairwise orthogonal manner and used for acquiring the acceleration in three directions;

and in the initial correction step, performing trigonometric operation according to the gravitational acceleration in three directions to determine the current holding position of the drumstick.

3. The percussion music intelligent education system based on motion detection auxiliary recognition according to claim 2, characterized in that the number of the gyro sensors is three, three gyro sensors are arranged in an orthogonal manner in pairs for obtaining rotation in three directions, and three gyro sensors correspond to the three acceleration sensors;

and respectively carrying out multiplication calculation with the gravity acceleration in the three directions acquired in the initial correction step according to the rotation amount components in the three directions acquired by the three gyroscope sensors to obtain the current gravity acceleration.

4. The intelligent percussion music education system based on motion detection and auxiliary recognition according to claim 3 is characterized in that the speed curve is obtained by performing integral operation according to the acceleration of the drumstick after gravity is removed, and the position of the drumstick relative to the initial correction step is obtained according to the speed time; and based on the position the stick is held, the position of the head of the stick is obtained.

5. The percussion music intelligent education system based on motion detection aided recognition according to claim 1, wherein the data processing process of the host computer further comprises: determining the state of the drum stick according to the instantaneous change of the position of the head of the drum stick, wherein the state of the drum stick comprises translation, downward striking and rebounding;

the scoring step comprises: and giving a music score according to the tone and the strength of the percussion instrument and the state of the drumstick.

6. The percussion music intelligent education system based on motion detection aided recognition according to claim 1, wherein the data processing process of the host computer further comprises: obtaining the force application point of the drum stick according to the proportion of the acceleration sensor and the gyroscope sensor, wherein the force application point of the drum stick comprises a wrist and the whole body;

the scoring step comprises: and giving a music score according to the tone and the strength of the percussion instrument, the state of the drumstick and the force point of the drumstick.

7. The system of claim 1, wherein in the audio feature extraction step, the audio features are obtained by extracting the ADSR envelope variation in the time domain and the MFCC variation in the frequency domain from the audio data.

8. The percussion music intelligent education system based on motion detection and auxiliary recognition as claimed in claim 1 is characterized in that the accuracy of percussion is judged according to the tone and strength obtained in real time and compared with a preset percussion music score, so as to obtain the score of music.

9. The percussion music intelligent education system based on motion detection aided recognition according to claim 1, wherein the training process of the percussion recognition neural network model is specifically as follows:

and acquiring training data, wherein the training data comprises model input data and an actual striking action result, loading the training data into a pre-established striking recognition neural network model, performing model training until a preset training stopping condition is reached, and acquiring the trained striking recognition neural network model.

10. The intelligent percussion music education system based on motion detection aided recognition according to claim 9, wherein the percussion recognition neural network model adopts a BP neural network.

Technical Field

The invention relates to the technical field of percussion music recognition, in particular to a percussion music intelligent education system based on motion detection auxiliary recognition.

Background

The existing percussion music identification adopts the method that pronunciation of percussion music is collected through audio, parameters such as a short-time power spectrum and zero crossing rate of the audio are obtained through analysis of the audio, and the audio initially belongs to the musical instrument. And then, obtaining the types and the hitting points of the musical instruments by using the dynamic time warping and comparing the characteristic parameters (MFCC) of various musical instruments and by methods of distance editing, neural network learning and the like.

For example, the invention with publication number CN110310666A discloses a musical instrument identification method and system based on SE convolutional network, the method includes the following steps: preprocessing data to be identified, and converting an audio file to be identified into an autocorrelation spectrogram to be identified; identifying data to be identified, inputting an autocorrelation spectrum chart to be identified into a pre-constructed musical instrument identification model for identification, and obtaining an output result matrix; and (4) analyzing the musical instruments, namely integrating and analyzing the output result matrix of the musical instrument identification model into a musical instrument label represented by a natural language.

The existing identification method has the defects that the identification rate of the system is low, the error of the identified percussion time of the musical instrument is large, the sound of the small musical instrument is submerged by loud sound, and the percussion action error cannot be indicated.

Disclosure of Invention

The invention aims to overcome the defects that the prior art has low recognition rate and large error of recognized percussion time of musical instruments, and provides an intelligent percussion education system based on motion detection and auxiliary recognition.

The purpose of the invention can be realized by the following technical scheme:

the utility model provides a percussion music intelligence educational system based on supplementary discernment of motion detection, includes drum stick, shelf drum and host computer, install acceleration sensor and gyroscope sensor on the drum stick, acceleration sensor and the equal communication connection of gyroscope sensor the host computer, the host computer still is used for gathering the audio data of shelf drum, the data processing process of host computer includes following step:

an initial correction step: at a static moment before the music begins, obtaining a gravity acceleration value according to a detection value of the acceleration sensor, and determining the current position held by the drum stick;

identifying the strength of the striking coordinate: when the drum stick acts, acquiring the rotation amount of the drum stick according to the gyroscope sensor, calculating the current gravitational acceleration according to the gravitational acceleration value acquired in the initial correction step, then subtracting the current gravitational acceleration from the data acquired by the current acceleration sensor to obtain the acceleration of the drum stick after gravity is removed, acquiring the position of the head of the drum stick according to the position held by the drum stick, and finally calculating the striking coordinate strength of the drum stick according to the momentum theorem;

audio characteristic extraction: extracting audio features according to the audio data;

identifying percussion music: loading the audio features and the striking coordinate strength of the drumstick into a pre-established and trained striking recognition neural network model to obtain the tone and strength of the percussion instrument,

grading: and giving a music score according to the tone and the strength of the percussion instrument.

Furthermore, the number of the acceleration sensors is three, and the three acceleration sensors are arranged in an orthogonal manner in pairs and are used for acquiring accelerations in three directions;

and in the initial correction step, performing trigonometric operation according to the gravitational acceleration in three directions to determine the current holding position of the drumstick.

Furthermore, the number of the gyroscope sensors is three, three gyroscope sensors are arranged in a pairwise orthogonal manner and used for acquiring rotation amounts in three directions, and the three gyroscope sensors correspond to the three acceleration sensors in position;

and respectively carrying out multiplication calculation with the gravity acceleration in the three directions acquired in the initial correction step according to the rotation amount components in the three directions acquired by the three gyroscope sensors to obtain the current gravity acceleration.

Further, integral operation is carried out according to the acceleration of the drum stick after gravity is removed, a speed curve is obtained, and the position of the drum stick relative to the drum stick in the initial correction step is obtained according to speed time; and based on the position the stick is held, the position of the head of the stick is obtained.

Further, the data processing process of the host further includes: determining the state of the drum stick according to the instantaneous change of the position of the head of the drum stick, wherein the state of the drum stick comprises translation, downward striking and rebounding;

the scoring step comprises: and giving a music score according to the tone and the strength of the percussion instrument and the state of the drumstick.

Further, the data processing process of the host further includes: obtaining the force application point of the drum stick according to the proportion of the acceleration sensor and the gyroscope sensor, wherein the force application point of the drum stick comprises a wrist and the whole body;

the scoring step comprises: and giving a music score according to the tone and the strength of the percussion instrument, the state of the drumstick and the force point of the drumstick.

Further, in the audio feature extraction step, the audio feature is obtained by extracting the ADSR envelope variation of the time domain and the MFCC variation feature of the frequency domain in the audio data.

And further, according to the tone and the strength acquired in real time, comparing the tone and the strength with a preset striking music score, and judging the accuracy of striking so as to obtain the score of the music.

Further, the training process of the strike recognition neural network model specifically includes:

and acquiring training data, wherein the training data comprises model input data and an actual striking action result, loading the training data into a pre-established striking recognition neural network model, performing model training until a preset training stopping condition is reached, and acquiring the trained striking recognition neural network model.

Further, the strike recognition neural network model adopts a BP neural network.

Compared with the prior art, the invention has the following advantages:

(1) the recognition rate is improved: by reconstructing the percussion position, through the parameters of the percussion position and combining the audio frequency, the recognition rate of the percussion instrument and the percussion coordinate strength of the system can be improved;

(2) the striking time is accurate: the acceleration of the drumstick after gravity is removed is calculated through the data of the acceleration sensor and the gyroscope sensor, the hitting action can be directly obtained, and the accurate hitting time is obtained;

(3) and (3) adding manipulation identification: the states of the striking techniques and the force positions of the striking can be obtained through the acceleration and the data analysis of the gyroscope, and the method is favorable for education assistance.

Drawings

Fig. 1 is a schematic diagram of a hardware structure of an intelligent percussion education system based on motion detection assisted recognition according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a percussion instrument timbre and force identification process of an intelligent percussion education system based on motion detection-assisted identification according to an embodiment of the present invention;

in the figure, 1 is a main unit, 2 is a drum stick, and 3 is a recorder.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example 1

As shown in fig. 1 and fig. 2, this embodiment provides a percussion music intelligent education system based on motion detection auxiliary recognition, including drumstick 2, drum kit and host computer 1, install acceleration sensor and gyroscope sensor on the drumstick, acceleration sensor and gyroscope sensor all communicate and connect the host computer, connect through bluetooth communication in this embodiment, the host computer still is used for gathering the audio data that has the drum kit, gather audio data through recorder 3 in this embodiment, the data processing process of host computer includes the following steps:

an initial correction step: at a static moment before the music begins, acquiring a gravity acceleration value according to a detection value of an acceleration sensor, and determining the current position held by a drum stick;

identifying the strength of the striking coordinate: when the drum rod acts, acquiring the rotation amount of the drum rod according to a gyroscope sensor, calculating the current gravitational acceleration according to the gravitational acceleration value acquired in the initial correction step, then subtracting the current gravitational acceleration from the data acquired by the current acceleration sensor to obtain the acceleration of the drum rod after the gravity is removed, acquiring the position of the head of the drum rod according to the position held by the drum rod, and finally calculating the striking coordinate strength of the drum rod according to the momentum theorem;

audio characteristic extraction: extracting audio features according to the audio data;

identifying percussion music: loading the audio features and the striking coordinate strength of the drumstick into a pre-established and trained striking recognition neural network model to obtain the tone and strength of the percussion instrument,

grading: and giving a score of the music according to the tone and the strength of the percussion instrument.

As a preferred embodiment, the number of the acceleration sensors is three, and the three acceleration sensors are arranged in an orthogonal manner in pairs and used for acquiring accelerations in three directions;

performing trigonometric operation according to the gravitational acceleration in three directions in the initial correction step to determine the current holding position of the drum stick; the position held by the stick at rest is conveniently determined by the acceleration of gravity in three directions.

Further, as a preferred embodiment, the number of the gyro sensors is three, three gyro sensors are orthogonally arranged in pairs and used for acquiring rotation amounts in three directions, and the three gyro sensors correspond to the positions of the three acceleration sensors;

according to the rotation amount components in three directions acquired by the three gyroscope sensors, multiplying the rotation amount components in three directions with the gravitational acceleration in three directions acquired in the initial correction step respectively to obtain the current gravitational acceleration; the number and the positions of the gyroscope sensors correspond to those of the acceleration sensors, so that component calculation is facilitated, and the calculation accuracy is improved.

The striking coordinate strength identification step specifically comprises the steps of carrying out integral operation according to the acceleration of the drumstick after gravity is removed, obtaining a speed curve, and obtaining the position of the drumstick relative to the initial correction step according to speed time; and based on the position the stick is held, the position of the head of the stick is obtained.

As a preferred embodiment, the data processing process of the host further includes: determining the state of the drum stick according to the instantaneous change of the position of the head of the drum stick, wherein the state of the drum stick comprises translation, downward striking and rebounding;

the scoring step comprises the following steps: and giving a music score according to the tone and the strength of the percussion instrument, the state of the drumstick and the force point of the drumstick.

The state of the identified drumstick is added, and the state of the drumstick is also a factor of performance scoring, so that the accuracy of scoring results can be improved.

Further, as a preferred embodiment, the data processing process of the host further includes: obtaining the force application points of the drumsticks according to the proportion of the acceleration sensor and the gyroscope sensor, wherein the force application points of the drumsticks comprise wrists and the whole body;

the scoring step comprises the following steps: and giving a music score according to the tone and the strength of the percussion instrument, the state of the drumstick and the force point of the drumstick.

The method specifically comprises the following steps: and comparing the obtained timbre, the force, the state of the drum stick and the force point of the drum stick with a preset striking music score and a scoring standard, and judging the striking accuracy so as to obtain the score of the music.

The force point of the drum stick is further added, and the force point of the drum stick is also a factor of performance scoring, so that the accuracy of scoring results can be further improved.

As a preferred embodiment, in the audio feature extraction step, the audio feature is obtained by extracting ADSR envelope variation in a time domain and MFCC variation feature in a frequency domain in the audio data.

In this embodiment, the training process of the strike recognition neural network model specifically includes:

the method comprises the steps of obtaining training data, loading the training data into a pre-established striking recognition neural network model, carrying out model training until a preset training stopping condition is reached, obtaining the trained striking recognition neural network model, wherein the training data comprise model input data and an actual striking action result, and the striking recognition neural network model adopts a BP neural network.

A combination of the above preferred embodiments can provide an optimal embodiment, which will be described in detail below.

The embodiment provides a percussion music intelligent education system based on motion detection auxiliary recognition, which consists of a host, a drumstick and a common drum set, wherein the host can directly record and analyze data. The intelligent drum stick is improved on the drum stick of percussion music, the gyroscope sensor and the acceleration sensor are added into the drum stick and are installed at the rear-end hand holding position, and the wireless transmission chip transmits acceleration information data of the gyroscope to the host at regular time.

The detailed description is as follows:

1. and (3) correction:

before the music begins, a static moment is found, when the three gyroscope sensors are zero, the drum stick is in a static state, and the system only has gravity acceleration. According to the principle that the three acceleration sensors are synthesized into exactly one gravity acceleration, the three acceleration sensors are synthesized into exactly the same gravity acceleration through the physical triangular force. And when the two components are consistent, performing trigonometric operation on the resultant force and the three component forces to obtain the current holding position of the drum stick.

The clock error is corrected by correcting the time difference of the transmission by wireless time.

2. Collecting:

and acquiring data of the acceleration sensor and the gyroscope sensor at regular time to obtain the state of the drumstick, and transmitting the state to the host through a wireless system. The host computer directly collects audio data.

3. Identification:

and reconstructing the position of the drumstick according to the audio data characteristic value, the acceleration sensor value and the gyroscope sensor value.

Calculating the rotation amount of the drum stick according to the data change of the three gyroscope sensors;

calculating the components of the current gravitational acceleration in three axial directions according to the rotation amount and the corrected gravitational acceleration component;

calculating the acceleration of the drumstick for removing the gravity according to the components of the current gravity acceleration in the three axial directions and the values of the three acceleration sensors;

solving a speed curve according to the acceleration integral of the gravity removal, and calculating the relative correction time position of the drum rod through speed time;

obtaining the position of the head of the drum stick through a gyroscope sensor and the installation position of the gyroscope sensor, and calculating the striking force according to the momentum theorem;

calculating the state of the drum stick according to the instantaneous change of the head of the drum stick, and performing translation, downward striking and rebounding;

according to the speed ratio of the gyroscope sensor and the acceleration sensor, the force application point, the wrist and the whole body of the drumstick are obtained;

identifying the percussion instrument and strength through a neural network according to the state of the drumstick and the audio frequency;

extracting audio features, extracting ADSR envelope variation of a time domain and MFCC variation features of a frequency domain, and taking the ADSR envelope variation and MFCC variation features as the features of the audio to participate in identification;

algorithm training:

the BP neural network is adopted, and an audio frequency and coordinate force data and an actual striking action result are input into a training algorithm through an existing known example. And obtaining the striking recognition neural network model through a large amount of data learning.

The identification process comprises the following steps:

and (3) adopting a BP neural network, identifying a neural network model according to the learnt striking, and inputting audio features and striking coordinate strength to obtain the tone and strength of the percussion instrument.

4. And (3) scoring:

the music is scored according to the force point of the drumstick, the rhythm of the striking state and the accuracy rate of the striking, and the insufficiency of the playing is indicated.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.