阅读说明：本技术 音乐和弦识别方法及装置、电子设备、存储介质 (Music chord identification method and device, electronic equipment and storage medium ) 是由 蒋慧军 徐伟 杨艾琳 姜凯英 肖京 于 2020-11-25 设计创作，主要内容包括：本申请涉及人工智能技术领域,具体提供了一种音乐和弦识别方法,该方法包括：针对待识别音乐和弦的音乐数据,依次提取音乐数据中含有的各个音符对应的音符信息,并基于各个音符对应的音符信息构建各个音符的二维矩阵表示；根据各个音符的二维矩阵表示提取各个音符对应的音符特征；基于各个音符对应的音符特征,分别从不同的和弦功能识别维度识别各个音符对应的和弦特征；将识别得到的同一音符在不同的和弦功能识别维度上的和弦特征进行组合,得到各个音符对应的和弦组合特征,并将各个音符对应的和弦组合特征所构成的特征序列作为音乐数据对应的音乐和弦识别结果。本申请能够基于数字音符识别音乐数据在和弦功能上的信息。(The application relates to the technical field of artificial intelligence, and particularly provides a music chord identification method, which comprises the following steps: sequentially extracting note information corresponding to each note contained in the music data aiming at the music data of the music chord to be identified, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note; extracting note characteristics corresponding to the notes according to the two-dimensional matrix representation of the notes; identifying the chord characteristics corresponding to the notes from different chord function identification dimensions respectively based on the note characteristics corresponding to the notes; and combining the chord features of the same note on different chord function identification dimensions to obtain chord combination features corresponding to the notes, and taking a feature sequence formed by the chord combination features corresponding to the notes as a music chord identification result corresponding to the music data. The information of the music data on the chord function can be identified based on the digital notes.)

1. A method for musical chord identification, comprising:

sequentially extracting note information corresponding to each note contained in the music data aiming at the music data of the music chord to be identified, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note;

extracting note characteristics corresponding to the notes according to the two-dimensional matrix representation of the notes;

identifying the chord features corresponding to the notes from different chord function identification dimensions respectively based on the note features corresponding to the notes;

and combining the chord features of the same note on the different chord function identification dimensions to obtain chord combination features corresponding to the notes, and taking a feature sequence formed by the chord combination features corresponding to the notes as a music chord identification result corresponding to the music data.

2. The method according to claim 1, wherein the data format of the music data is a musical instrument data interface format; extracting note information corresponding to each note contained in the music data in sequence, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note, including:

sequentially extracting note pitches and note durations of notes contained in the music data, and taking the note pitches and the note durations as note information corresponding to the notes;

and constructing a two-dimensional matrix representation of each note by taking the note pitch as a longitudinal element in a two-dimensional matrix and the note duration as a transverse element in the two-dimensional matrix.

3. The method of claim 1, wherein extracting the note feature corresponding to each note according to the two-dimensional matrix representation of each note comprises:

acquiring a two-dimensional matrix representation sequence formed by two-dimensional matrix representation of each note;

and inputting the two-dimensional matrix representation sequence into a feature extraction model, and acquiring a note feature sequence output by the feature extraction model aiming at the two-dimensional matrix representation sequence, wherein the note feature sequence contains note features corresponding to all notes.

4. The method of claim 1, wherein identifying the chord characteristics corresponding to the respective notes from different chord function identification dimensions based on the note characteristics corresponding to the respective notes comprises:

acquiring a note characteristic sequence formed by note characteristics corresponding to the notes;

and respectively inputting the note characteristic sequences into a plurality of preset chord function identification models to obtain chord characteristics obtained by identifying and processing each note characteristic in the note characteristic sequences by each chord function identification model from different chord function identification dimensions.

5. The method of claim 4, wherein the chord function identification dimension includes at least a chord key dimension, and a chord index dimension that collectively act on the chord function representation of the music data.

6. The method of claim 1, further comprising:

acquiring a data set used for training a feature extraction model and a plurality of chord function identification models, wherein the data set contains a plurality of music data to be trained;

dividing each piece of music data to be trained into a first music data segment, a second music data segment and a third music data segment, forming a training data set based on the first music data segment corresponding to each piece of music data to be trained, forming a test data set based on the second music data segment corresponding to each piece of music data to be trained, and forming a verification data set based on the third music data segment corresponding to each piece of music data to be trained;

and training the feature extraction model and the plurality of chord function identification models according to the training data set, the test data set and the verification data set so as to extract note features corresponding to all notes in the music data of the music chord to be identified based on the trained feature extraction model, and identifying the chord features corresponding to all the notes from different chord function identification dimensions based on the trained plurality of chord function identification models.

7. The method according to claim 5, wherein the input signals of the plurality of chord function recognition models are output signals of the feature extraction model; training the feature extraction model and the plurality of chord function recognition models according to the training data set, the test data set and the verification data set, including:

training the feature extraction model according to the training data set, the test data set and the verification data set;

after the trained feature extraction model is obtained, training the plurality of chord function recognition models according to the training data set, the test data set and the verification data set and the output signals of the trained feature extraction model;

and training loss values corresponding to the training of the chord function recognition models respectively, and finishing the training of the chord function recognition models when the sum of the training loss values corresponding to the chord function recognition models is smaller than a loss threshold value.

8. A musical chord recognition apparatus, comprising:

the musical note information processing module is configured to sequentially extract musical note information corresponding to each musical note contained in music data aiming at the music data of the music chord to be identified, and construct two-dimensional matrix representation of each musical note based on the musical note information corresponding to each musical note;

a note feature extraction module configured to extract note features corresponding to the notes according to the two-dimensional matrix representation of the notes;

the chord feature identification module is configured to identify the chord features corresponding to the notes from different chord function identification dimensions respectively based on the note features corresponding to the notes;

and the recognition result acquisition module is configured to combine the chord features of the same note in the different chord function recognition dimensions to obtain chord combination features corresponding to the notes, and use a feature sequence formed by the chord combination features corresponding to the notes as a music chord recognition result corresponding to the music data.

9. An electronic device, comprising:

a memory storing computer readable instructions;

a processor to read computer readable instructions stored by the memory to perform the method of any of claims 1-7.

10. A computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor of a computer, cause the computer to perform the method of any one of claims 1-7.

Technical Field

The application relates to the technical field of artificial intelligence, in particular to a music chord identification method and device, electronic equipment and a computer readable storage medium.

Background

With the continuous development of computer information technology, the computer technology is more and more widely applied in artistry. For example, the number of western classical music works is huge, and the classical music learning can be more convenient and the propagation of classical music can have greater potential by establishing an automatic classical music analysis system.

When music analysis is performed on classical music, the chord performance of the classical music generally needs to be analyzed, but at present, the analysis on the chord performance of the classical music is realized based on the analysis on chord symbols, and chord properties such as chord root of a chord can be obtained, but the analysis mode cannot obtain information on the chord function of the classical music.

Disclosure of Invention

In order to solve the above technical problem, embodiments of the present application provide a method and an apparatus for identifying a musical chord, an electronic device, and a computer-readable storage medium.

The technical scheme provided by the embodiment of the application comprises the following steps:

a music chord identification method, comprising: sequentially extracting note information corresponding to each note contained in the music data aiming at the music data of the music chord to be identified, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note; extracting note characteristics corresponding to the notes according to the two-dimensional matrix representation of the notes; identifying the chord features corresponding to the notes from different chord function identification dimensions respectively based on the note features corresponding to the notes; and combining the chord features of the same note on the different chord function identification dimensions to obtain chord combination features corresponding to the notes, and taking a feature sequence formed by the chord combination features corresponding to the notes as a music chord identification result corresponding to the music data.

In one exemplary embodiment, the data format of the music data is an instrument data interface format; extracting note information corresponding to each note contained in the music data in sequence, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note, including: sequentially extracting note pitches and note durations of notes contained in the music data, and taking the note pitches and the note durations as note information corresponding to the notes; and constructing a two-dimensional matrix representation of each note by taking the note pitch as a longitudinal element in a two-dimensional matrix and the note duration as a transverse element in the two-dimensional matrix.

In an exemplary embodiment, extracting the note feature corresponding to each note according to the two-dimensional matrix representation of each note includes: acquiring a two-dimensional matrix representation sequence formed by two-dimensional matrix representation of each note; and inputting the two-dimensional matrix representation sequence into a feature extraction model, and acquiring a note feature sequence output by the feature extraction model aiming at the two-dimensional matrix representation sequence, wherein the note feature sequence contains note features corresponding to all notes.

In an exemplary embodiment, identifying the chord characteristics corresponding to the respective notes from different chord function identification dimensions respectively based on the note characteristics corresponding to the respective notes includes: acquiring a note characteristic sequence formed by note characteristics corresponding to the notes; and respectively inputting the note characteristic sequences into a plurality of preset chord function identification models to obtain chord characteristics obtained by identifying and processing each note characteristic in the note characteristic sequences by each chord function identification model from different chord function identification dimensions.

In one exemplary embodiment, the chord function identification dimension includes at least a chord key dimension, and a chord index dimension that collectively act on the chord function representation of the music data.

In one exemplary embodiment, the method further comprises: acquiring a data set used for training a feature extraction model and a plurality of chord function identification models, wherein the data set contains a plurality of music data to be trained; dividing each piece of music data to be trained into a first music data segment, a second music data segment and a third music data segment, so that a training data set is formed based on the first music data segment corresponding to each piece of music data to be trained, a test data set is formed based on the second music data segment corresponding to each piece of music data to be trained, and a verification data set is formed based on the third music data segment corresponding to each piece of music data to be trained; and training the feature extraction model and the plurality of chord function identification models according to the training data set, the test data set and the verification data set so as to extract note features corresponding to all notes in the music data of the music chord to be identified based on the trained feature extraction model, and identifying the chord features corresponding to all the notes from different chord function identification dimensions based on the trained plurality of chord function identification models.

In one exemplary embodiment, the input signals of the plurality of chord function recognition models are all output signals of the feature extraction model; training the feature extraction model and the plurality of chord function recognition models according to the training data set, the test data set and the verification data set, including: training the feature extraction model according to the training data set, the test data set and the verification data set; after the trained feature extraction model is obtained, training the plurality of chord function recognition models according to the training data set, the test data set and the verification data set and the output signals of the trained feature extraction model; and training loss values corresponding to the training of the chord function recognition models respectively, and finishing the training of the chord function recognition models when the sum of the training loss values corresponding to the chord function recognition models is smaller than a loss threshold value.

A musical chord identification apparatus comprising: the musical note information processing module is configured to sequentially extract musical note information corresponding to each musical note contained in music data aiming at the music data of the music chord to be identified, and construct two-dimensional matrix representation of each musical note based on the musical note information corresponding to each musical note; a note feature extraction module configured to extract note features corresponding to the notes according to the two-dimensional matrix representation of the notes; the chord feature identification module is configured to identify the chord features corresponding to the notes from different chord function identification dimensions respectively based on the note features corresponding to the notes; and the recognition result acquisition module is configured to combine the chord features of the same note in the different chord function recognition dimensions to obtain chord combination features corresponding to the notes, and use a feature sequence formed by the chord combination features corresponding to the notes as a music chord recognition result corresponding to the music data.

An electronic device comprising a processor and a memory, the memory having stored thereon computer readable instructions which, when executed by the processor, implement the musical chord identification method of any of the preceding claims.

A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a musical chord identification method as in any one of the preceding claims.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

in the technical scheme provided by the embodiment of the application, the chord characteristics of each note on different chord function identification dimensions are automatically extracted from the music data of the music chord to be identified based on an artificial intelligence mode, and then the chord characteristics of the same note on different chord function identification dimensions are combined, so that the finally obtained music chord identification result contains the chord information of each note on different chord function identification dimensions.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 is a flow diagram illustrating a method of music chord identification according to an exemplary embodiment;

FIG. 2 is a schematic diagram illustrating the structure of a music chord recognition model in accordance with an exemplary embodiment;

FIG. 3 is a flowchart illustrating a method of musical chord identification in accordance with another exemplary embodiment;

FIG. 4 is a block diagram illustrating a musical chord identification apparatus in accordance with an exemplary embodiment;

fig. 5 is a schematic diagram of a hardware structure of an electronic device according to an exemplary embodiment.

While certain embodiments of the present application have been illustrated by the accompanying drawings and described in detail below, such drawings and description are not intended to limit the scope of the inventive concepts in any manner, but are rather intended to explain the concepts of the present application to those skilled in the art by reference to the particular embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

As mentioned above, the analysis of the chord performance of classical music is implemented based on the analysis of chord symbols, and chord properties such as chord root of the chord can be obtained, but this analysis method cannot obtain information about the chord function of classical music.

In order to solve the problem, the present application provides a music chord identification scheme based on artificial intelligence, and in particular provides a music chord identification method, an apparatus, an electronic device and a computer readable storage medium. Based on the music chord identification scheme provided by the application, chord function information of each note in a chord mode, a chord tone property, a chord transposition and the like can be identified and obtained from music data of the music chord to be identified.

The music chord identification method, apparatus, electronic device and computer readable storage medium proposed by the present application will be described in detail with specific embodiments.

Referring to FIG. 1, FIG. 1 is a flow diagram illustrating a method for musical chord identification in accordance with an exemplary embodiment. The music chord identification method at least comprises steps S110 to S140, which are described in detail as follows:

step S110, sequentially extracting note information corresponding to each note contained in the music data for the music chord to be identified, and constructing a two-dimensional matrix representation of each note based on the note information corresponding to each note.

It should be noted that the chord referred to in the embodiments of the present application is a musical theory, and may be understood as a music melody with a certain musical interval relationship, and the definition of the chord is not described in detail herein.

The music data of the music chord to be identified refers to digital music obtained by coding by using a digital music standard format, and the music data records the music melody through the musical note and the digital control information. For example, the data format of the music data may be a Musical Instrument Data Interface (MIDI) format, where MIDI is a widely used music standard format, the music data format can be understood by a computer, specifically, the music data format includes instructions such as Musical notes and control participation, and almost all current music is produced based on the MIDI format.

Therefore, in the embodiment, based on the musical notes contained in the music data and the instructions such as the control parameters, the musical note information corresponding to each musical note can be sequentially extracted from the music data of the music chord to be identified. For example, the note information corresponding to each note may include the note pitch and note duration of each note, or may extract the required note information based on the actual scene, which is not limited herein.

Based on the note information extracted for each note in the present embodiment, a two-dimensional feature representation of each note can be constructed. As mentioned above, if the note pitch and note duration of each note are extracted for each note, the note pitch may be used as a longitudinal element in the two-dimensional matrix, and the note duration may be used as a transverse element in the two-dimensional matrix, or the note pitch may be used as a transverse element in the two-dimensional matrix, and the note duration may be used as a longitudinal element in the two-dimensional matrix, so as to construct a two-dimensional matrix representation of each note.

The two-dimensional matrix representation constructed by using the note pitch as a longitudinal element in a two-dimensional matrix and the note duration as a transverse element in the two-dimensional matrix can be understood as a coordinate system constructed by using the note pitch as a longitudinal coordinate and using time as a transverse coordinate, so that the two-dimensional matrix representation of each note can carry note information corresponding to the note, and further, the information related to the chord function of the music data is obtained based on the identification processing of the two-dimensional matrix representation of each note.

It should be noted that the music data related to the music chord to be identified in this embodiment may be a digital representation corresponding to any form of music, such as classical music and pop music, and this implementation does not limit the type of music to which this music data belongs.

And step S120, extracting note characteristics corresponding to the notes according to the two-dimensional matrix representation of the notes.

In this embodiment, in order to obtain information related to the chord function of music data, it is further necessary to extract feature information on the two-dimensional matrix representation, for example, feature information of each note on the chord function, such as a chord Key (usually denoted as Key), a chord Key (usually denoted as Quality), and a chord index (usually denoted as Inversion), so as to obtain a note feature corresponding to each note.

For example, to obtain note features corresponding to each note conveniently, a two-dimensional matrix representation sequence formed by two-dimensional matrix representation of each note may be obtained, then the two-dimensional matrix representation sequence is input into the feature extraction model, and a note feature sequence output by the feature extraction model for the two-dimensional matrix representation sequence is obtained, where the obtained note feature sequence includes note features corresponding to each note. For example, the feature extraction model may be a machine learning model, such as a Long Short-Term Memory (LSTM) model, a Bi-directional Long Short-Term Memory (Bi-LSTM) model, etc., without limitation. However, in general, the Bi-LSTM model has a better feature information extraction effect than the LSTM model, and therefore, in an actual application scenario, the Bi-LSTM model may be selected to extract note features corresponding to each note.

In some other embodiments, feature information on chord functions such as chord key style, chord key performance, chord transposition and the like may also be extracted from the two-dimensional matrix representation of each note based on some note feature extraction algorithms to obtain note features corresponding to each note, which is not limited herein.

Step S130, based on the note characteristics corresponding to each note, identifying the chord characteristics corresponding to each note from different chord function identification dimensions, respectively.

As described above, in this embodiment, in step S120, the feature information of each note on the chord functions such as the chord key, and the chord index is further extracted for the two-dimensional matrix representation, so as to obtain the note feature corresponding to each note, and thus, the chord feature corresponding to each note can be identified from the chord function identification dimensions different from the chord key dimension, and the chord index dimension.

It should be noted that the chord mode dimension, the chord key dimension, and the chord index dimension described above collectively act on the chord function representation of the music data, and in a scenario of analyzing the music data (for example, classical music), these chord function identification dimensions are mutually influenced, so that the chord features corresponding to the respective notes need to be identified from different chord function identification dimensions respectively.

Of course, based on different music data analysis requirements, the chord features corresponding to each note may also be identified from other chord function identification dimensions, such as a chord dominance (usually expressed as pre.

In some embodiments, a note feature sequence formed by note features corresponding to each note may be obtained, and then the note feature sequence is respectively input into a plurality of preset chord function identification models, so as to obtain note features obtained by identifying, by each chord function identification model, each note in the note feature sequence from different chord function identification dimensions. It should be noted that different chord function identification models correspond to different chord function identification dimensions, and therefore, in this embodiment, based on the note characteristics corresponding to each note, the process of identifying the chord characteristics corresponding to each note from the different chord function identification dimensions may be performed separately or simultaneously as multiple subtasks for extracting chord characteristics, which is not limited herein.

For example, the plurality of chord function recognition models preset in this embodiment are machine learning models, for example, CRF (Conditional Random Field) models may be adopted, and different CRF models are used for constraining the recognition process of the models for each note feature in the note feature sequence from different chord function recognition dimensions.

In some embodiments, if the extraction of the note feature corresponding to each note according to the two-dimensional matrix representation of each note in the music data is implemented based on the aforementioned feature extraction model, the note feature sequence input into the chord function identification model may be an output signal of the aforementioned feature extraction model. That is, in these embodiments, the chord features of the notes included in the music data in different chord function identification dimensions are automatically extracted in the form of a model combining the LSTM model and the CRF model, and the two models are dependent and constrained in the chord feature extraction process, so that the obtained chord features are more accurate.

In addition, in this embodiment, based on the note characteristics corresponding to each note, the process of identifying the chord characteristics corresponding to each note from different chord function identification dimensions may also be implemented according to a chord characteristic identification algorithm in different chord function identification dimensions, which is not limited in this embodiment.

Step S140, the chord features of the same note obtained by identification on different chord function identification dimensions are combined to obtain a chord combination feature corresponding to each note, and a feature sequence formed by the chord combination features corresponding to each note is used as a music chord identification result corresponding to the music data.

As mentioned above, the chord characteristics of each note in the music data of the music chord to be identified in step S130 in different chord function identification dimensions are obtained, and the different chord function identification dimensions are mutually influenced, so as to facilitate accurate and comprehensive analysis of the music data, in this embodiment, the chord characteristics of the same note in the different chord function identification dimensions obtained by identification are combined, so as to comprehensively represent the characteristic information of each note in the chord function based on the obtained chord combination characteristics.

For example, the chord features of the same note in different chord function identification dimensions are combined, and the chord features in different chord function identification dimensions may be spliced according to a specified splicing sequence to obtain the chord combination feature.

Or in some embodiments, the process of combining the chord features of the same note in different chord function identification dimensions may specifically be a process of storing the chord features of the same note in different chord function identification dimensions in association with the corresponding note. Specifically, a unique note identifier may be assigned to each note included in the music data, for example, corresponding note identifiers may be assigned according to an arrangement sequence of each note in the note data, then an association relationship between each note and a chord feature of the note in a different chord function identification dimension may be constructed, and the chord feature of each note in the different chord function identification dimension may be stored in the database based on the constructed association relationship. When the music chord identification result corresponding to the music data needs to be displayed, for example, when a user needs to check the music chord identification result corresponding to the music data, the chord characteristics of each note in different chord function identification dimensions can be called from the database according to the association relationship of each note to be displayed, so that the method based on the embodiment is very convenient for displaying the music chord identification result.

Therefore, in the music chord identification result obtained in the embodiment, the chord information of each note in the music data containing the music chord to be identified on different chord function identification dimensions is contained, that is, the embodiment can identify the related characteristic information of the music data on the chord function based on the digital note, and solves the problem that the information on the chord function of classical music cannot be obtained in the prior art.

FIG. 2 is a schematic diagram illustrating the structure of a music chord recognition model according to an exemplary embodiment. The music chord identification model is used for extracting chord characteristics of each note in different chord function identification dimensions aiming at each note contained in music data of music chords to be identified so as to automatically extract characteristic information of the music data in the chord function.

As shown in fig. 2, the exemplary music chord recognition model is composed of a feature extraction model 10 and a plurality of chord function recognition models, wherein the plurality of chord function recognition models may include chord function recognition models 21-23 shown in fig. 2, wherein the chord function recognition models 21-23 respectively extract the relevant feature information of the music data from different chord function recognition dimensions.

Illustratively, the feature extraction model 10 includes a Bi-LSTM network and a fully connected network. The Bi-LSTM network is used for extracting note characteristics of a two-dimensional matrix representation sequence input into the Bi-LSTM network, wherein the two-dimensional matrix representation sequence is a sequence formed by two-dimensional matrix representation of each note contained in music data of a music chord to be identified, and the two-dimensional matrix representation of each note is constructed on the basis of note information corresponding to each note. As shown in fig. 2, the Bi-LSTM network extracts corresponding note features from the two-dimensional matrix representations "s 1, s2, s3 … … sn" that are input thereto, respectively, and performs full-link processing on the obtained note features corresponding to the notes via a full-link network to obtain a note feature sequence formed by the note features corresponding to the notes, and outputs the obtained note feature sequence to each chord function recognition model. Each chord function recognition model may be a machine learning model obtained by training based on a CRF model.

The chord function recognition model 21 performs recognition processing on the note features of each note included in the note feature sequence from the chord key dimension based on the note feature sequence output by the feature extraction model 10, and outputs a first chord feature sequence formed by the chord features of each note in the chord key dimension.

The chord function recognition model 22 performs recognition processing on the note features of each note included in the note feature sequence from the chord key dimension based on the note feature sequence output by the feature extraction model 10, and outputs a second chord feature sequence formed by the chord features of each note in the chord key dimension.

The chord function recognition model 23 performs recognition processing on the note features of each note contained in the note feature sequence from the chord index dimension based on the note feature sequence output by the feature extraction model 10, and outputs a third chord feature sequence formed by the chord features of each note in the chord index dimension.

It can be seen that, in the first chord feature sequence, the second chord feature sequence and the third chord feature sequence output by each chord function model, each note in the music data of the music chord to be identified has a chord feature in a different chord function identification dimension. In an exemplary application scenario, chord features of the same note identified by each chord function model in different chord function identification dimensions may be combined to obtain chord combination features corresponding to each note, and a feature sequence formed by the chord combination features corresponding to each note is used as a music chord identification result corresponding to music data.

It should be noted that, for each note contained in the music data of the music chord to be identified, the detailed process of extracting the chord features of each note in different chord function identification dimensions by the music chord identification model shown in fig. 2 is referred to the specific description of the embodiment shown in fig. 1, and is not described herein again.

In some exemplary embodiments, to ensure that the music chord recognition model shown in fig. 2 has a good recognition effect, the music chord recognition model shown in fig. 2 may be trained by using the training process shown in fig. 3.

As shown in FIG. 3, in an exemplary embodiment, training the music chord recognition model includes steps S210 through S230, which are detailed as follows:

step S210, a data set for training the feature extraction model and the chord function recognition models is obtained, where the data set contains a plurality of music data to be trained.

It should be noted that, in order to obtain a better training effect of the music chord recognition model, in the data set for training the feature extraction model and the plurality of chord function recognition models, each piece of music data to be trained is usually a digital music representation corresponding to a piece of complete music, for example, may be music data corresponding to a complete song.

Step S220, dividing each piece of music data to be trained into a first music data segment, a second music data segment, and a third music data segment, so that the first music data segment corresponding to each piece of music data to be trained forms a training data set, the second music data segment corresponding to each piece of music data to be trained forms a test data set, and the third music data segment corresponding to each piece of music data to be trained forms a verification data set.

The dividing manner of different music data segments for each piece of music data to be trained may include multiple manners, for example, music data to be trained may be divided into different music data segments with equal duration on average according to duration; or key data positions can be positioned in the music data to be trained, and music data segments can be divided based on the positioned key data positions to correspondingly obtain corresponding music data segments such as prelude, verse, refrain and the like; alternatively, the data information corresponding to the repeated melody may be contained in the different divided music data segments, which is not limited in this embodiment.

It should be noted that, the dividing manner of different music data segments for each music data to be trained may be the same, so that the first music data segments contained in the training data set obtained in this embodiment have feature consistency, so as to perform targeted training on the music chord recognition model. In addition, the music chord recognition model is trained by adopting the test data set and the verification data set, so that the music chord recognition model has a better training effect.

And step S230, training the feature extraction model and the chord function recognition models according to the training data set, the test data set and the verification data set.

In this embodiment, a training data set may be used to perform first round training on the feature extraction model and the plurality of chord function recognition models, after the trained feature extraction model and the plurality of chord function recognition models are trained, a test data set may be used to perform second round training on the feature extraction model and the plurality of chord function recognition models, where the second round training aims to test the model effects of the feature extraction model and the plurality of chord function recognition models trained in the first round, and further optimize the model effects of the feature extraction model and the plurality of chord function recognition models based on the test results. And after the second round of training, verifying the model effect by using the verification data set, and if the model effect obtained by verification is not good, further optimizing the model effects of the feature extraction model and the plurality of chord function recognition models.

Because the training data set, the test data set and the verification data set are obtained by dividing and summarizing music data segments for the same data set, the music chord recognition model is trained based on the training data set, the test data set and the verification data set, and equivalently, the music chord recognition model is trained for multiple times based on the same training data, so that the training effect of the music chord recognition model can be improved, the training data quantity of the music chord recognition model can be increased to a greater extent, and the training effect of the music chord recognition model can be further improved.

Further, in the music chord recognition model shown in fig. 2, the chord function recognition models 21 to 23 share the output signal of the feature extraction model 10, that is, the model parameters of the feature extraction model 10 are shared between the chord function recognition models 21 to 23. Thus, in some embodiments, the feature extraction model may be trained from a training dataset, a test dataset, and a validation dataset; after the trained feature extraction model is obtained, training each chord function identification model according to a training data set, a test data set and a verification data set and an output signal of the trained feature extraction model; and then training corresponding training loss values for each chord function recognition model respectively, and finishing the training for the plurality of chord function recognition models when the sum of the training loss values corresponding to each chord function recognition model is less than a loss threshold value. In the embodiment, each chord function recognition model is trained based on the trained feature extraction model, so that the training rate of each chord function recognition model can be improved.

In other embodiments, the combination of the feature extraction model and any one of the chord function recognition models may also be trained according to a training data set, a test data set, and a verification data set, and in each training process, the model parameters in the feature extraction model and the chord function recognition model are updated correspondingly based on the training loss value, so that the training mode proposed by this embodiment is adopted to train the feature extraction model multiple times, so that the music chord recognition model has a better training effect.

FIG. 4 is a block diagram illustrating a musical chord identification apparatus according to an exemplary embodiment, as shown in FIG. 4, including:

the note information processing module 410 is configured to sequentially extract note information corresponding to each note contained in the music data for the music chord to be identified, and construct a two-dimensional matrix representation of each note based on the note information corresponding to each note; a note feature extraction module 420 configured to extract note features corresponding to the notes according to the two-dimensional matrix representation of the notes; the chord feature identification module 430 is configured to identify the chord features corresponding to the notes from different chord function identification dimensions respectively based on the note features corresponding to the notes; the identification result obtaining module 440 is configured to combine the chord features of the same identified note in different chord function identification dimensions to obtain chord combination features corresponding to each note, and use a feature sequence formed by the chord combination features corresponding to each note as a music chord identification result corresponding to music data.

The music chord identification device shown in the embodiment can identify the information of the music data on the chord function based on the digital notes, and can solve the problem that the information on the chord function of classical music cannot be obtained in the prior art.

In another exemplary embodiment, the data format of the music data is an instrument data interface format, and the note information processing module 410 includes:

a note information acquisition unit configured to sequentially extract note pitches and note durations of respective notes contained in the music data, and to use the note pitches and the note durations as note information corresponding to the respective notes; and the two-dimensional matrix representation construction unit is configured to construct a two-dimensional matrix representation of each note by taking the pitch of the note as a longitudinal element in the two-dimensional matrix and the duration of the note as a transverse element in the two-dimensional matrix.

In another exemplary embodiment, the note feature extraction module 420 includes:

a first sequence acquisition unit configured to acquire a two-dimensional matrix representation sequence constituted by two-dimensional matrix representations of respective notes; and the first model processing unit is configured to input the two-dimensional matrix representation sequence into the feature extraction model, and acquire a note feature sequence output by the feature extraction model aiming at the two-dimensional matrix representation sequence, wherein the note feature sequence contains note features corresponding to all notes.

In another exemplary embodiment, the chord characteristic identification module 430 includes:

a second sequence acquisition unit configured to acquire a note feature sequence constituted by note features corresponding to respective notes; and the second model processing unit is configured to input the note feature sequences into a plurality of preset chord function identification models respectively so as to obtain chord features obtained by identifying and processing the note features in the note feature sequences by the chord function identification models from different chord function identification dimensions.

In another exemplary embodiment, the chord function identification dimension includes at least a chord key dimension, and a chord index dimension that collectively act on the chord function representation of the music data.

In another exemplary embodiment, the apparatus further comprises:

the data set acquisition module is configured to acquire a data set used for training the feature extraction model and the chord function recognition models, and the data set contains a plurality of music data to be trained; the data set processing module is configured to divide each piece of music data to be trained into a first music data segment, a second music data segment and a third music data segment, form a training data set based on the first music data segment corresponding to each piece of music data to be trained, form a test data set based on the second music data segment corresponding to each piece of music data to be trained, and form a verification data set based on the third music data segment corresponding to each piece of music data to be trained; and the model training module is configured to train the feature extraction model and the plurality of chord function identification models according to the training data set, the test data set and the verification data set so as to extract the note features corresponding to the notes in the music data of the music chord to be identified based on the trained feature extraction model, and identify the chord features corresponding to the notes from different chord function identification dimensions based on the trained plurality of chord function identification models.

In another exemplary embodiment, the input signals of the plurality of chord function recognition models are output signals of a feature extraction model, and the model training module includes:

the feature extraction model training unit is configured to train a feature extraction model according to a training data set, a test data set and a verification data set; the chord function recognition model training unit is configured to train a plurality of chord function recognition models according to a training data set, a test data set and a verification data set and output signals of the trained feature extraction models after the trained feature extraction models are obtained; and the training monitoring unit is configured to respectively perform training loss values corresponding to the training for each chord function recognition model, and when the sum of the training loss values corresponding to each chord function recognition model is smaller than a loss threshold value, the training for the plurality of chord function recognition models is ended.

It should be noted that the apparatus provided in the foregoing embodiment and the method provided in the foregoing embodiment belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiment, and is not described again here.

In an exemplary embodiment, the present application further provides an electronic device comprising a processor and a memory, the memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of musical chord identification as previously described.

Fig. 5 is a schematic diagram of a hardware structure of an electronic device according to an exemplary embodiment.

It should be noted that the electronic device is only an example adapted to the application and should not be considered as providing any limitation to the scope of use of the application. The electronic device is also not to be construed as requiring reliance on, or necessity of, one or more components of the exemplary electronic device illustrated in fig. 5.

The hardware structure of the electronic device may have a large difference due to the difference of configuration or performance, as shown in fig. 5, the electronic device includes: a power supply 510, an interface 530, at least one memory 550, and at least one Central Processing Unit (CPU) 570.

The power supply 510 is used for providing an operating voltage for each hardware device on the electronic device.

The interface 530 includes at least one wired or wireless network interface 531, at least one serial-to-parallel conversion interface 533, at least one input/output interface 535, and at least one USB interface 537, etc. for communicating with external devices.

The memory 550 is used as a carrier for resource storage, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon include an operating system 551, an application program 553, data 555, etc., and the storage manner may be a transient storage manner or a permanent storage manner.

The operating system 551 is used to manage and control hardware devices and application programs 553 on the electronic device, so as to implement the calculation and processing of the mass data 555 by the central processing unit 570, which may be Windows server, Mac OS XTM, unix, linux, or the like. The application programs 553 are computer programs that perform at least one particular task on the operating system 551, and may include at least one module (not shown in FIG. 5) that may each include a sequence of computer-readable instructions for an electronic device. Data 555 may be http protocol data stored on disk, etc.

Central processor 570 may include one or more processors and is configured to communicate with memory 550 via a bus for computing and processing mass data 555 in memory 550.

As described in detail above, an electronic device to which the present application is applied will read a series of computer readable instructions stored in the memory 550 by the CPU 570 to complete the music chord identification method described in the previous embodiments.

Furthermore, the present application can also be implemented by hardware circuitry or by a combination of hardware circuitry and software instructions, and thus the implementation of the present application is not limited to any specific hardware circuitry, software, or combination of both.

In an exemplary embodiment, the present application further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the music chord identification method as described above.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.