阅读说明：本技术 乐器演奏点的确定及节奏评分的方法 (Method for determining musical instrument playing point and scoring rhythm ) 是由 覃建博 于 2019-10-11 设计创作，主要内容包括：本发明公开了一种乐器演奏点的确定及节奏评分的方法,包含步骤：S1、获取音频演奏数据；S2、将音频演奏数据分割成固定长度的片段,并按照先后顺序命名,进行短时傅里叶变换,得到相应名称的固定比例的频谱图；S3、将频谱图输入目标检测框架；S4、选择是否根据确定性分数删除若干分数低于一预设值的边界框；S5、根据文件名重组边界框序列；S6、设定边界框的两个横坐标为X1和X2,得到用户演奏点的时间序列；S7、将模板音频数据进行步骤S1-S6操作,得到模板演奏点的时间序列；S8、使用动态规划进行点匹配；S9、进行评分。音频数据在经过短时傅里叶变换后输出的数据使用色彩梯度生成图像,视觉上体现为颜色较深,判断此演奏点是噪音还是乐器演奏音。(The invention discloses a method for determining musical instrument playing points and scoring rhythms, which comprises the following steps: s1, acquiring audio performance data; s2, dividing the audio performance data into segments with fixed length, naming the segments according to the sequence, and performing short-time Fourier transform to obtain a spectrogram with a fixed proportion and a corresponding name; s3, inputting the spectrogram into a target detection framework; s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores; s5, reorganizing the bounding box sequence according to the file name; s6, setting two abscissa of the bounding box as X1 and X2 to obtain a time sequence of the playing points of the user; s7, carrying out operations of steps S1-S6 on the template audio data to obtain a time sequence of the playing points of the template; s8, performing point matching by using dynamic programming; and S9, scoring. The data output after the audio data is subjected to short-time Fourier transform generates an image by using color gradient, the image is visually embodied to be darker in color, and whether the playing point is noise or musical instrument playing sound is judged.)

1. A method for determining a musical instrument playing point, comprising the steps of:

s1, acquiring audio performance data of the user by using hardware equipment;

s2, dividing the audio performance data into segments with fixed length, naming the segments according to the sequence, and performing short-time Fourier transform to obtain a spectrogram with a fixed proportion and a corresponding name;

s3, inputting the spectrogram into a target detection framework, and outputting a bounding box and a corresponding certainty score;

s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores;

s5, recombining the bounding box sequence according to the file name to obtain a real bounding box in the time sequence;

s6, setting the two abscissa of the bounding box as X1 and X2, the real performance point X3 is (X1+ X2)/2, thereby obtaining the time series of the user performance points.

2. A method for tempo scoring of a musical instrument playing point, comprising the steps of:

s1, acquiring audio performance data of the user by using hardware equipment;

s2, dividing the audio performance data into segments with fixed length, naming the segments according to the sequence, and performing short-time Fourier transform to obtain a spectrogram with a fixed proportion and a corresponding name;

s3, inputting the spectrogram into a target detection framework, and outputting a bounding box and a corresponding certainty score;

s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores;

s5, recombining the bounding box sequence according to the file name to obtain a real bounding box in the time sequence;

s6, setting the two abscissa of the bounding box as X1 and X2, and obtaining the time series of the playing points of the user by the real playing point X3 ═ X1+ X2)/2;

s7, carrying out operations of steps S2-S6 on the template audio data to obtain a time sequence of the playing points of the template;

s8, performing point matching by using dynamic programming, and searching the corresponding relation between the user playing point and the template playing point;

and S9, scoring according to the preset matching relation.

Technical Field

The invention relates to the field of electronic musical instruments, in particular to a method for determining a musical instrument playing point and scoring rhythm.

Background

The existing method for determining the musical instrument playing point generally adopts methods such as difference value processing, Fourier transform and the like, the method cannot accurately obtain the musical instrument playing point of a segment with a fast rhythm, experiences are poor in a noisy environment, and meanwhile, the method is difficult to detect the musical instrument playing point with a small volume.

Disclosure of Invention

The invention mainly solves the technical problems in the prior art, and provides a method for determining a musical instrument playing point and scoring rhythm.

The technical problem of the invention is mainly solved by the following technical scheme:

a method for determining a musical instrument playing point, comprising the steps of:

s1, acquiring audio performance data of the user by using hardware equipment;

s2, dividing the audio performance data into segments with fixed length, naming the segments according to the sequence, and performing short-time Fourier transform to obtain a spectrogram with a fixed proportion and a corresponding name;

s3, inputting the spectrogram into a target detection framework, and outputting a bounding box and a corresponding certainty score;

s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores;

s5, recombining the bounding box sequence according to the file name to obtain a real bounding box in the time sequence;

s6, setting the two abscissas of the bounding box as X1 and X2, and then the real playing point X3 is defined as the real playing point X3

(X1+ X2)/2, thereby obtaining a time series of user performance points.

A method for tempo scoring of a musical instrument playing point, comprising the steps of:

s1, acquiring audio performance data of the user by using hardware equipment;

s2, dividing the audio performance data into segments with fixed length, naming the segments according to the sequence, and performing short-time Fourier transform to obtain a spectrogram with a fixed proportion and a corresponding name;

s3, inputting the spectrogram into a target detection framework, and outputting a bounding box and a corresponding certainty score;

s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores;

s5, recombining the bounding box sequence according to the file name to obtain a real bounding box in the time sequence;

s6, setting the two abscissas of the bounding box as X1 and X2, and then the real playing point X3 is defined as the real playing point X3

(X1+ X2)/2, thereby obtaining a time series of user performance points;

s7, carrying out operations of steps S2-S6 on the template audio data to obtain a time sequence of the playing points of the template;

s8, performing point matching by using dynamic programming, and searching the corresponding relation between the user playing point and the template playing point;

and S9, scoring according to the preset matching relation.

The method for determining the musical instrument playing points and scoring the rhythm has the following advantages that: since each musical instrument performance causes frequency change, the data output after the audio data is subjected to short-time fourier transform generates an image by using color gradient, and the image is visually represented as a darker color in the region. The invention uses the target detection technology to determine the dark place, and carries out two classifications to judge whether the playing point is noise or musical instrument playing sound.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a flow chart of the method for determining musical instrument playing points and scoring rhythms according to the present invention;

FIG. 2 is a diagram of audio waveforms of a user before segmentation;

FIG. 3 is a waveform of the audio of FIG. 2 after the audio is divided into parts according to a fixed length;

FIG. 4 is a waveform of the divided audio of FIG. 3 amplified to a fixed scale without a short-time Fourier transform;

FIG. 5 is a spectral plot of the waveform of FIG. 4 after a short-time Fourier transform;

FIG. 6 is a graph of the output spectrum of the waveform of FIG. 5 after passing through the target detection framework;

FIG. 7 is a schematic diagram of segment hash point matching.

Detailed Description

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and the scope of the present invention will be more clearly and clearly defined.

As shown in fig. 1, the method for determining the musical instrument playing point and scoring the rhythm includes the steps of:

s1, acquiring the audio performance data of the user using the hardware device, as shown in fig. 2;

s2, dividing the audio performance data into segments of fixed length and naming them in sequence, as shown in fig. 3. Then, performing short-time fourier transform to obtain a spectrogram with a fixed ratio of a corresponding name, as shown in fig. 4 and 5;

s3, inputting the spectrogram into an object detection framework, and outputting bounding boxes and corresponding certainty scores, as shown in fig. 6, where each box in fig. 6 is a performance point;

s4, selecting whether to delete a plurality of bounding boxes with the scores lower than a preset value according to the certainty scores;

s5, recombining the bounding box sequence according to the file name to obtain a real bounding box in the time sequence;

s6, setting the two abscissas of the bounding box as X1 and X2, and then the real playing point X3 is defined as the real playing point X3

(X1+ X2)/2, thereby obtaining a time series of user performance points;

s7, carrying out operations of steps S2-S6 on the template audio data to obtain a time sequence of the playing points of the template;

s8, performing point matching by using dynamic programming, and searching the corresponding relation between the user playing point and the template playing point;

and S9, scoring according to the preset matching relation.

The output of the spectrogram after passing through the object detection framework is further described below.

In fig. 6, each box is a performance POINT in the format of (X1, Y1, X2, Y2, POINT), where X1 and X2 respectively represent estimated values of the start POINT and end POINT of the abscissa,y1 and Y2 represent the performance strength, and the real performance time point X3 is (X1+ X2)/2, so that a time series u [ u ] of the user performance points can be obtained₁,u₂,u₃,u₄,u₅]. Meanwhile, the template data is subjected to the same steps to obtain a time series t [ t ] of the template performance point₁,t₂,t₃,t₄,t₅]。

As shown in FIG. 7, when matching is performed, u₁Matching t₁，u₂Matching t₂,u₃If no corresponding point is found on the time series t of the template playing points, u₃For the user in u₂And u₄Multiple played tones in between. See again t₄，t₄If no corresponding point is found on the time series u of the playing points of the user, the user can play the corresponding point on the time series u₄And u₅One tone less than the other is played. Finally, the score obtained by the user is the average score after the scores of the multi-performance tones and the few-performance tones are removed,

namely, it is

Where avg is taken as the average and abs is taken as the absolute value, where n is 5.

It should be noted that, because each musical instrument plays the musical instrument to cause the frequency change, the data output after the audio data is subjected to the short-time fourier transform uses the color gradient to generate the image, which is visually embodied as that the color of this region is darker, for example, the position with darker gray in fig. 5 is the playing point. The invention uses the target detection technology to determine the dark place, and carries out two classifications to judge whether the playing point is noise or musical instrument playing sound.

Without being limited thereto, any changes or substitutions that are not thought of through the inventive work should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope defined by the claims.