阅读说明：本技术 课堂对话的评价方法、系统和存储介质 (Classroom conversation evaluation method, system and storage medium ) 是由 宋宇 雷顺威 于 2021-08-10 设计创作，主要内容包括：本发明公开了一种课堂对话的评价方法、系统和存储介质,可应用于大数据处理技术领域。方法包括以下步骤：获取课堂视频数据或课堂音频数据转换得到的文本信息；获取对所述文本信息进行第一编码标注后的第一数据,以及获取对所述文本信息进行第二编码标注后的第二数据,所述第一编码标注和所述第二编码标注的编码均属于预设编码体系；确定所述第一数据和所述第二数据满足预设要求,采用所述第一数据和所述第二数据对预设编码标注模型进行训练；采用训练后的所述预设编码标注模型对所述文本信息进行编码标注,并根据编码标注结果生成课堂评价结果。本发明能减少人工标注数据的工作量,提高课堂对话的评价效率和评价的准确度。(The invention discloses an evaluation method, an evaluation system and a storage medium for classroom conversation, which can be applied to the technical field of big data processing. The method comprises the following steps: acquiring text information obtained by converting classroom video data or classroom audio data; acquiring first data obtained after first coding labeling is carried out on the text information, and acquiring second data obtained after second coding labeling is carried out on the text information, wherein codes of the first coding labeling and the second coding labeling both belong to a preset coding system; determining that the first data and the second data meet preset requirements, and training a preset coding labeling model by adopting the first data and the second data; and coding and labeling the text information by adopting the trained preset coding and labeling model, and generating a classroom evaluation result according to a coding and labeling result. The invention can reduce the workload of manual data annotation and improve the evaluation efficiency and the evaluation accuracy of classroom conversation.)

1. A classroom conversation evaluation method is characterized by comprising the following steps:

acquiring text information obtained by converting classroom video data or classroom audio data;

acquiring first data obtained after first coding labeling is carried out on the text information, and acquiring second data obtained after second coding labeling is carried out on the text information, wherein codes of the first coding labeling and the second coding labeling both belong to a preset coding system;

determining that the first data and the second data meet preset requirements, and training a preset coding labeling model by adopting the first data and the second data;

and coding and labeling the text information by adopting the trained preset coding and labeling model, and generating a classroom evaluation result according to a coding and labeling result.

2. The method according to claim 1, wherein the obtaining of the text information converted from the classroom video data or classroom audio data comprises:

acquiring classroom video data or classroom audio data as third data;

extracting dialogue data in the third data;

converting the dialogue data into text information.

3. The method according to claim 2, wherein the converting the dialogue data into text information comprises:

acquiring a speaking body and speaking information in the dialogue data;

taking the speaking main body and the corresponding speaking information as a first corpus;

transforming the speaking main body in the first corpus to obtain a second corpus;

and generating text information according to the first corpus and the second corpus.

4. The method as claimed in claim 3, wherein the number of codes of each of the first corpus or the second corpus comprises at least one.

5. The method as claimed in claim 1, wherein the codes in the predetermined coding system include basic knowledge, personal information, analysis, induction, migration and innovation, response and construction, recognition, question and instruction, nine categories, 15 first-level indicators and 39 second-level indicators.

6. The method according to claim 1, wherein the determining that the first data and the second data satisfy a predetermined requirement, and the training of a predetermined coding labeling model using the first data and the second data comprises:

comparing a first encoding label of the first data with a second encoding label of the second data;

and determining that the similarity of all the first coding labels and the second coding labels meets a preset requirement, and training a preset coding label model by adopting the first data and the second data.

7. The method according to claim 1, wherein the preset coding labeling model comprises an embedding layer, a convolutional neural network layer, a bidirectional long-term memory neural network layer and an output layer; the method for coding and labeling the text information by adopting the trained preset coding and labeling model comprises the following steps:

converting the text information into continuous vectors by adopting the embedding layer;

extracting local features from the continuous vectors by using the convolutional neural network layer;

extracting global features from the continuous vectors by adopting the bidirectional long-time and short-time memory neural network layer;

and coding and labeling the text information according to the local features and the global features.

8. The method of claim 7, wherein the embedding layer comprises a language representation model; the converting the text information into continuous vectors using the embedding layer includes:

and converting the text information into continuous vectors by adopting the language representation model.

9. A classroom conversation evaluation system, comprising:

at least one memory for storing a program;

at least one processor for loading the program to perform a method of rating a classroom conversation as claimed in any one of claims 1-7.

10. A storage medium in which a computer-executable program is stored, the computer-executable program being executed by a processor for implementing the evaluation method of a classroom conversation as claimed in any one of claims 1 to 7.

Technical Field

The invention relates to the technical field of big data processing, in particular to an evaluation method, an evaluation system and a storage medium for classroom conversation.

Background

Classroom conversation is a core means for implementing classroom interaction communication, is a main gripper for guiding thinking and researching problems, and also bears the important function of enlightening thinking, so that the essence of developing high-quality classroom teaching is to enhance the effectiveness of classroom conversation. In the related art, currently, the following disadvantages exist for the classroom conversation data processing mode: firstly, a universal and scientific classroom conversation evaluation system is lacked, evaluation indexes with different types are provided in different modes, and the names, definitions, meanings and types of the indexes are inconsistent, so that large-scale classroom conversation analysis and comparison are difficult to implement; secondly, in the conventional method, quantitative behavior data analysis of a multi-emphasis surface is performed on evaluation of classroom conversation, most of the quantitative behavior data pay attention to behavior data of shallow levels such as conversation frequency, conversation subject, conversation time and the like, less attention is paid to behavior data reflecting learning quality behind languages or behaviors, deep mining and analysis are lacked to cognitive and thinking characteristics implicit in the behavior data, and the quantitative behavior data analysis is difficult to effectively correspond to a culture target of education and teaching; thirdly, the current classroom conversation analysis method mainly adopts manual words-by-words or sentence-by-sentence labeling, but only the manual labeling speed is low, the average time of each class classroom conversation labeling is 7-8 hours, and the phenomenon of low internal reliability is easy to occur due to different levels of the labeling persons, so that the requirement of large-scale classroom teaching evaluation is difficult to meet.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides an evaluation method, an evaluation system and a storage medium for classroom conversation, which can improve the evaluation efficiency and the evaluation accuracy of classroom conversation.

In one aspect, an embodiment of the present invention provides an evaluation method for a classroom conversation, including the following steps:

acquiring text information obtained by converting classroom video data or classroom audio data;

acquiring first data obtained after first coding labeling is carried out on the text information, and acquiring second data obtained after second coding labeling is carried out on the text information, wherein codes of the first coding labeling and the second coding labeling both belong to a preset coding system;

determining that the first data and the second data meet preset requirements, and training a preset coding labeling model by adopting the first data and the second data;

and coding and labeling the text information by adopting the trained preset coding and labeling model, and generating a classroom evaluation result according to a coding and labeling result.

In some embodiments, the obtaining of text information obtained by converting the classroom video data or classroom audio data includes:

acquiring classroom video data or classroom audio data as third data;

extracting dialogue data in the third data;

converting the dialogue data into text information.

In some embodiments, said converting said dialogue data into text information comprises:

acquiring a speaking body and speaking information in the dialogue data;

taking the speaking main body and the corresponding speaking information as a first corpus;

transforming the speaking main body in the first corpus to obtain a second corpus;

and generating text information according to the first corpus and the second corpus.

In some embodiments, each of the encoded numbers of the first corpus or the second corpus includes at least one.

In some embodiments, the coding in the predetermined coding scheme includes nine major categories of basic knowledge, personal information, analysis, induction, migration and innovation, response and construction, recognition, challenge and guidance, and fifteen primary codes, namely, question basic knowledge, response basic knowledge, question personal information, response personal information, analytical question, analytical response, inductive question, inductive response, migration and innovation question, migration and innovation response, recognition, challenge and guidance, and 39 secondary indexes.

In some embodiments, the determining that the first data and the second data satisfy the preset requirement, and training a preset coding labeling model using the first data and the second data includes:

comparing a first encoding label of the first data with a second encoding label of the second data;

and determining that the similarity of all the first coding labels and the second coding labels meets a preset requirement, and training a preset coding label model by adopting the first data and the second data.

In some embodiments, the preset coding labeling model comprises an embedding layer, a convolutional neural network layer, a bidirectional long-term memory neural network layer and an output layer; the method for coding and labeling the text information by adopting the trained preset coding and labeling model comprises the following steps:

converting the text information into continuous vectors by adopting the embedding layer;

extracting local features from the continuous vectors by using the convolutional neural network layer;

extracting global features from the continuous vectors by adopting the bidirectional long-time and short-time memory neural network layer;

and coding and labeling the text information according to the local features and the global features.

In some embodiments, the embedding layer includes a language representation model; the converting the text information into continuous vectors using the embedding layer includes:

and converting the text information into continuous vectors by adopting the language representation model.

In another aspect, an embodiment of the present invention provides an evaluation system for a classroom conversation, including:

at least one memory for storing a program;

and the at least one processor is used for loading the program to execute the evaluation method of the classroom conversation.

In another aspect, an embodiment of the present invention provides a storage medium, in which a computer-executable program is stored, and the computer-executable program is executed by a processor to implement the above-mentioned evaluation method for a classroom conversation.

The classroom conversation evaluation method provided by the embodiment of the invention has the following beneficial effects:

the embodiment firstly obtains text information obtained by converting classroom video data or classroom audio data, then obtains first data obtained by performing first coding labeling on the text information, and second data obtained by performing second coding labeling on the text information, then when the first data and the second data meet preset requirements, the first data and the second data are adopted to train a preset coding labeling model, then the trained preset coding labeling model is adopted to perform coding labeling on the text information, and classroom evaluation results are generated according to the coding labeling results, the embodiment labels the text information in a uniform coding labeling mode, thereby obtaining precondition applied to large-scale classroom dialogue analysis and comparison, the text information is subjected to coding labeling through the trained preset coding labeling model, and workload of manual labeling data is reduced, meanwhile, behavior data reflecting learning quality is obtained, so that the evaluation efficiency and the evaluation accuracy of classroom conversation are improved.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The invention is further described with reference to the following figures and examples, in which:

fig. 1 is a flowchart of an evaluation method of a classroom conversation according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of high-frequency evaluation indexes of classroom conversation according to an embodiment of the present invention;

FIG. 3 is a diagram of a default coding annotation model according to an embodiment of the invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, the meaning of a plurality is one or more, the meaning of a plurality is two or more, and the above, below, exceeding, etc. are understood as excluding the present numbers, and the above, below, within, etc. are understood as including the present numbers. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.

In the description of the present invention, unless otherwise explicitly defined, terms such as arrangement, connection and the like should be broadly construed, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the detailed contents of the technical solutions.

In the description of the present invention, reference to the description of the terms "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like means that a particular feature or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, or characteristics, described may be combined in any suitable manner in any one or more embodiments or examples.

A classroom conversation refers to a language communication activity in which a teacher and students spread around an educational teaching objective in a classroom, including an online classroom and an offline classroom. The development of a classroom conversation requires the presence of a problem originator and at least one other person to respond to the problem. Initiating interaction Initiation-responding Response-Feedback or Evaluation Feedback/Evaluation (IRF or IRE) is a typical operating mode of classroom conversations, i.e. a teacher or student initiates an interaction in the form of a question, leads to thinking questions, and then others respond, and the teacher or student gives Feedback and evaluates the answer. In the conversation, the improvement of the number of the questions and the positivity of the responses is really worthy of encouragement, but the quality of the conversation is the core value and is also the key for enhancing the classroom efficiency. A high quality classroom conversation should have five core attributes, namely collectivity, reciprocity, support, constructivity and purposiveness. The high-performance dialog should start with open questions, aim at knowledge construction, and take explanation, analysis, reasoning, induction and meta-cognition as main expression forms. The social culture theory is an important theoretical basis for classroom conversation research, and emphasizes that cognitive abilities are developed through interaction in a social culture environment, wherein language is an important cognitive development medium and a carrier of thinking development. Under the guidance of the theory, a large number of demonstration research results come from the beginning, the important role of classroom conversation on the development of students is proved from multiple angles, the classroom conversation is beneficial to exchanging and sharing information, the collision of multiple viewpoints is promoted, and deep understanding is achieved; the method is beneficial to improving the criticizing and thinking resistance, cultivating high-level thinking such as logic analysis, summary induction, abstract reasoning and the like, and is beneficial to enhancing the innovation capability and the problem solving capability. The existing evaluation mode for classroom conversation has the problems of non-uniform evaluation standards, need of manually marking a large amount of data and the like.

Based on this, the embodiment of the invention provides an evaluation method, an evaluation system and a storage medium for classroom conversation. The text information is marked in a uniform coding and marking mode, so that the precondition applied to large-scale classroom conversation analysis and comparison is obtained, the text information is coded and marked through a trained preset coding and marking model, the workload of manual marking data is reduced, behavior data reflecting learning quality is obtained, and accordingly the evaluation efficiency and the evaluation accuracy of classroom conversation are improved.

The present invention will be described in detail with reference to the accompanying drawings.

Referring to fig. 1, an embodiment of the present invention provides an evaluation method for a classroom conversation. The method is applied to a preset platform system, and a preset Coding system (CI-PCD) for Coding and labeling text information and a preset Coding and labeling model for automatically labeling the text are stored in the preset platform system in advance. The code of the preset coding system is an evaluation index consisting of basic knowledge, personal information, analysis, induction, migration and innovation, response and construction, approval, question and guidance. These codes can be communicated indexes extracted from journal literature and evaluation models in the field of classroom teaching in recent 20 years by adopting literature measurement. The document metering method is a modern scientific research method which takes a document system and document metering characteristics as research objects, adopts metering research methods such as mathematics, statistics and the like to research the distribution structure, the quantity relationship and the change rule of documents and further discusses the structure, the characteristics and the rule of the corresponding field, and can provide scientific and objective summary for the research condition in the field from a macroscopic view. The literature selection is mainly from a Web of Science (WOS) database, the WOS database is authoritative for international research, SSCI and SCI journal literature is included, the quality of the literature is high, and the wide international recognition is achieved. The nine evaluation indexes have high frequency, and can embody the target of knowledge exploration and thinking progression, as shown in fig. 2. The abscissa in fig. 2 represents the class conversation code type that has been screened out from the previous study and the ordinate represents the frequency of occurrence of the code, and it can be well verified from fig. 2 that several types of codes, such as Analysis (Analysis), basic knowledge (Prior-knowledge), response and construction (update), migration and innovation (generation), and personal information (personal information), have higher importance, and therefore, they are included in the evaluation index of the present embodiment.

Specifically, the "basic knowledge" class has the functions of helping students obtain information, concepts, basic knowledge, facts, and learn basic methods and basic rules; the function of the teaching of the 'personal information' class is to guide students to express personal thought, emotion, viewpoint and feeling; the function of the 'analysis' type dialogue is to guide students to deeply analyze problems, improve deductive thinking capability and strengthen deep understanding; the induction type conversation has the functions of cultivating the ability of students to comprehensively and comprehensively see problems, improving the global thinking and finding the operation rule of things through comparison and connection; the functions of the 'migration and innovation' dialogue are supporting students to develop innovative thinking and improving the migration and application capability; the 'response and construction' type conversation aims at guiding students to listen to other people to speak and collaborate with people, and self-thinking is improved; the expression of 'recognition' can play a social role in linking up conversations, encouraging listening and promoting communication; the question-like conversation is beneficial to the culture of critical thinking, is the embodiment of actively participating in classroom learning, and is a means for exciting exploration and innovation; the "guide" words are targeted support and guidance provided by the teacher according to the learning progress and cognitive level of the students in the classroom teaching process. On the basis, the embodiment further distinguishes conversation bodies and conversation forms, namely a teacher body and a student body, and meanwhile each type of evaluation index is further divided into questions and responses, for example, the teacher presents analytical questions and the student presents analytical answers. Through combining three dimensions of classroom conversation content, main part and form, this embodiment can be better to standard education teaching target. Specifically, the contents of the preset coding system CI-PCD are shown in table 1:

TABLE 1

Specifically, as shown in fig. 1, the embodiment includes the following steps in the process of the embodiment:

and S11, acquiring text information obtained by converting the classroom video data or classroom audio data.

In the embodiment of the present application, the obtaining of the text information specifically includes obtaining classroom video data or classroom audio data as third data, then extracting dialogue data in the third data, and converting the dialogue data into the text information. Specifically, when the dialogue data is converted into the text information, the speaking body and the speaking information in the dialogue data may be obtained first, then the speaking body and the corresponding speaking information are used as the first corpus, the speaking body in the first corpus is transformed to obtain the second corpus, and then the text information is generated according to the first corpus and the second corpus. For example, the text arrangement of classroom conversation is performed according to the speaking subjects of teachers and students, the speaking of each subject is a corpus, the subjects are transformed to form a new corpus, and all the obtained corpora are used as text information.

S12, acquiring first data after the text information is subjected to first coding labeling, and acquiring second data after the text information is subjected to second coding labeling. And the codes of the first coding label and the second coding label both belong to a preset coding system.

In the embodiment of the application, after the text information is obtained, the text information is labeled sentence by sentence according to a CI-PCD coding system, and then which code in the table 1 the sentence belongs to is marked. Multiple types of codes can be selected from each corpus, and when multiple coded contents are related in one corpus, multiple codes can be selected. For example, a corpus is "classmates, please explain why can C be concluded with the a and B conditions with the knowledge we just learned? ", the corpus would be encoded as" basic knowledge questions "and" analytic questions ". The coding labeling process can be completed by manually selecting a corresponding coding button in the platform. For example, two experts are selected to select corresponding encoding buttons for encoding the same text information in the platform, the first data corresponds to the data after the first expert performs encoding and labeling, and the second data corresponds to the data after the second expert performs encoding and labeling.

And S13, determining that the first data and the second data meet the preset requirements, and training a preset coding annotation model by adopting the first data and the second data.

In the embodiment of the application, after the first data and the second data of the marked information to be coded are obtained, the codes of the two data are compared, when the consistency of the comparison reaches more than 0.9, the inconsistent content is verified, and the verified code corpus can be sent to a machine for learning. The first coding labels of the first data and the second coding labels of the second data are compared, the similarity of all the first coding labels and the second coding labels is determined to meet the preset requirement, and the first data and the second data are adopted to train a preset coding label model.

And S14, coding and labeling the text information by adopting the trained preset coding and labeling model, and generating a classroom evaluation result according to the coding and labeling result.

In the embodiment of the present application, as shown in fig. 3, the preset coding labeling model includes an embedding layer, a convolutional neural network layer, a bidirectional long-and-short-term memory neural network layer, and an output layer; wherein the embedding layer comprises a BERT language characterization model. The step of coding and labeling the text information by adopting the trained preset coding and labeling model is realized by the following modes:

converting the text information into continuous vectors by adopting a language representation model in the embedded layer; extracting local features from the continuous vectors by adopting a convolutional neural network layer; extracting global features from the continuous vectors by adopting a bidirectional long-time memory neural network layer; and coding and labeling the text information according to the local features and the global features. The language representation model Bidirectional Encoder retrieval from transformations (BERT) can preprocess texts, is regarded as a general language understanding model trained on a large text corpus, can effectively represent classroom dialogue texts as continuous vectors, is applied to downstream computing tasks of a neural network, and is suitable for input formats of deep learning modes. And then carrying out model training, wherein a neural network is widely applied to deep information learning of text data in a classification task, a data set is randomly divided into two subsets of a training set and a testing set for model training and model evaluation, a hybrid neural network model CNN-BilSTM combining a convolutional neural network and a bidirectional long-short term memory network is adopted, the hybrid model can simulate a human memory system and is used for acquiring local features and global features from the training data set so as to mine hidden semantic information in a dialogue text, the convolutional neural network extracts dialogue local features, the bidirectional long-short term memory neural network (BilSTM) serialization is favorable for extracting the global features, and further classroom dialogue quality is evaluated according to the extracted local features and the global features.

For example, the above embodiments are applied to a specific test procedure, which is specifically: experiments are carried out by adopting a real classroom teaching record of the middle and primary school stage of 600, which comprises three subjects of Chinese, mathematics and science. Wherein, 80% of the classroom dialogue corpus (11101 pieces) is used as the training corpus, and 20% (2776 pieces) of the classroom dialogue corpus is used as the test corpus. The specific corpus experimental data are shown in table 2:

TABLE 2

Coding large classes	Training data set	Test data set	Sum of
				Basic knowledge	3996	999	4995
Personal opinion expression	2151	537	2688
				Description of the analysis	2012	503	2515
Summary induction	817	204	1021
				Migration innovation	862	215	1077
Responsive construction	710	177	887
				Others (guidance for identity and question)	589	105	694

Applying the above experimental data to the proposed method of the present embodiment and the existing method, the results shown in table 3 were obtained:

TABLE 3

As can be seen from Table 3, the CNN + BilSTM used has the highest parameters and the best training effect. After training, the Precision rate, the recall rate and the F1 score of the machine automatic labeling text adopted by the embodiment all reach over 0.7, the Precision rate Precision is 0.8842, the recall rate is 0.8782, and the F1 score is 0.7531. The three parameters are widely used for evaluating the performance of the classification model, which means that the machine labeling has higher reliability and effectiveness, and the machine automatically labels the average time of 15 seconds per class (40 minutes), thereby greatly improving the analysis labeling speed and precision and achieving the level of large-scale application and analysis of classroom conversation.

Therefore, when the embodiment is applied to actual classroom evaluation, an important foundation is laid for developing large-scale classroom teaching comparison research and mining classroom conversation rules, mathematical teaching barriers between each section and each school are broken, data island effect is broken, and relatively uniform basis is provided for comparison and evaluation of classroom teaching levels. The effectiveness of the training and research activities of math teachers is improved, the training and research activities of teachers are usually based on subjective experiences of expert teachers or teaching and research personnel at present and have randomness and dispersity, scientific and prospective guidance and support are provided for the training and research activities, classroom teaching transformation is facilitated, interactive, heuristic and exploration type teaching is developed, and innovative practical talents are cultured.

The embodiment of the invention provides an evaluation system of classroom conversation, which comprises:

at least one memory for storing a program;

at least one processor for loading the program to perform the evaluation method of the classroom conversation of fig. 1.

The content of the embodiment of the method of the invention is all applicable to the embodiment of the system, the function of the embodiment of the system is the same as the embodiment of the method, and the beneficial effect achieved by the embodiment of the system is the same as the beneficial effect achieved by the method.

An embodiment of the present invention provides a storage medium in which a computer-executable program is stored, which, when executed by a processor, is used to implement the evaluation method of the classroom conversation shown in fig. 1.

The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and executed by the processor to cause the computer device to perform the method illustrated in fig. 1.

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention. Furthermore, the embodiments of the present invention and the features of the embodiments may be combined with each other without conflict.