阅读说明：本技术 一种口肌训练结合的多模态言语康复训练系统 (Multi-modal speech rehabilitation training system combining oral muscle training ) 是由 姚琤 郝玥 王章志 郑星临 李笑千 应放天 于 2021-07-06 设计创作，主要内容包括：本发明公开了一种口肌训练结合的多模态言语康复训练系统,包括人机交互设备,所述人机交互设备包括有数据采集单元、数据分析单元、数据对照单元、显示单元和数据库,所述数据采集单元与数据分析单元电性连接,所述数据分析单元与数据对照单元电性连接,所述数据对照单元与显示单元和数据库连接。本发明利用音频对照技术、智能推荐、模块化学习的方式,将发音练习与口肌训练交叉进行,结合以友好的人机交互界面引导激发听障儿童的学习兴趣,提升言语发音的准确度,并大大降低了言语训练的学习成本,有助于言语训练的普及性增强；根据学习的难易程度分为不同等级,有助于用户个性化选择更符合自身水平的训练内容。(The invention discloses a multi-modal speech rehabilitation training system combining oral muscle training, which comprises human-computer interaction equipment, wherein the human-computer interaction equipment comprises a data acquisition unit, a data analysis unit, a data comparison unit, a display unit and a database, the data acquisition unit is electrically connected with the data analysis unit, the data analysis unit is electrically connected with the data comparison unit, and the data comparison unit is connected with the display unit and the database. According to the invention, the pronunciation training and the oral muscle training are performed in a crossed manner by using an audio contrast technology, intelligent recommendation and a modular learning manner, and the learning interest of hearing-impaired children is stimulated by combining with a friendly human-computer interaction interface, so that the accuracy of speech pronunciation is improved, the learning cost of speech training is greatly reduced, and the popularization of speech training is enhanced; the training content is divided into different grades according to the difficulty degree of learning, and the training content which is more suitable for the user is selected in a personalized mode.)

1. The utility model provides a multi-modal speech rehabilitation training system that musculus oralis training combines, includes human-computer interaction equipment, its characterized in that: the human-computer interaction equipment comprises a data acquisition unit (1), a data analysis unit (2), a data comparison unit (3), a display unit (4) and a database (5), wherein the data acquisition unit (1) is electrically connected with the data analysis unit (2), the data analysis unit (2) is electrically connected with the data comparison unit (3), and the data comparison unit (3) is connected with the display unit (4) and the database (5).

2. The system of claim 1, wherein the system comprises: display element (4) are including setting up in display screen and the operation keyboard on human-computer interaction equipment surface, display element (4) are used for practising content, exercise action and exercise score, be provided with the USB interface on the human-computer interaction equipment.

3. The system of claim 1, wherein the system comprises: data acquisition unit (1) receives module (8) including sound collection module (6), image acquisition module (7) and gas sense, sound collection module (6) sets up the adapter on human-computer interaction equipment for the embedding, image acquisition module (7) are the camera that sets up in human-computer interaction equipment upper end, gas sense module (8) are the baroceptor of setting at human-computer interaction equipment front end.

4. The system of claim 1, wherein the system comprises: the data analysis unit (2) is used for sorting and analyzing the acquired data, and the analysis result is transmitted to the data comparison unit (3).

5. The system of claim 1, wherein the system comprises: the data comparison unit (3) comprises a voice comparison module (9), an image comparison module (10) and an air pressure comparison module (11).

6. The system of claim 5, wherein the system comprises: the voice comparison module (9) is used for comparing the audio collected by the sound collection module (6) with the standard audio in the database (5) and obtaining a training score according to the coincidence rate; the image comparison module (10) is used for comparing the mouth shape image acquired by the image acquisition module (7) with a standard mouth shape image in the database (5) and obtaining a training score according to the coincidence rate; the air pressure comparison module (11) is used for comparing the breath law acquired by the air sensing module (8) with the standard breath law in the database (5) and obtaining the training score according to the coincidence rate.

7. The method of using a system for multi-modal speech rehabilitation training in conjunction with oral muscle training according to any one of claims 1-6, wherein: comprises the following steps of (a) carrying out,

the method comprises the following steps: the training materials are imported into a database (5), and the database (5) comprises three difficult degrees of simple, complex and difficult training;

step two: a user selects the difficulty degree during training on a display screen of the human-computer interaction device, pronouncing training is carried out according to the display guidance of an interaction interface, and the audio frequency, the image and the exhalation air pressure information of the user are collected by using the data collection unit (1) during the pronouncing training;

step three: the data acquisition unit (1) transmits acquired data information to the data analysis unit (2), the data analysis unit (2) analyzes acquired audio, image and barometric pressure data and transmits an analysis result to the data comparison unit (3), the data comparison unit (3) compares training information of a user with standard pronunciation information in the database (5) and obtains a comprehensive score according to the coincidence rate of the training information of the user and the standard pronunciation information;

step four: if the comprehensive score is lower than 50 points, the user carries out targeted oral muscle training content;

step five: the user carries out targeted oral muscle training according to the guidance of a display screen on the human-computer interaction equipment, and a data acquisition unit (1) records the targeted oral muscle training process;

step six: after the targeted oral muscle training is finished, secondary pronunciation is carried out, and the data comparison unit (3) compares and scores pronunciation training information and standard pronunciation information until the pronunciation training score exceeds 50, so that the training is finished.

8. The method of using a multi-modal speech rehabilitation training system in combination with oral muscle training according to claim 7, wherein: the material for speech training of the hearing-impaired children in the first step comprises initial consonants, vowels, single syllables, phrase syllables and tones.

9. The method of using a multi-modal speech rehabilitation training system in combination with oral muscle training according to claim 7, wherein: and the targeted oral muscle training in the fourth step comprises pronunciation mouth shape, pronunciation phonetic symbols and gas training, and the training is carried out by selecting the items with lower scores according to each score of the first pronunciation training in the third step.

Technical Field

The invention relates to the technical field of deaf-mute children speech impairment rehabilitation, in particular to a multi-modal speech rehabilitation training system combining oral muscle training.

Background

According to the latest data of world health organization official networks, about 4.66 million people worldwide suffer from disabled hearing loss, and about 3400 ten thousand children. China is the world with the most hearing disabilities. With the further development of the current hearing aid technology, the wide use of hearing aids and the clinical technology of cochlear implants are remarkably improved, so that patients with different hearing impairment degrees can select proper hearing compensation equipment. Since the 2006 report of the union of chinese disabled people, the chinese government has been working on expanding and standardizing hearing rehabilitation services, and the hearing services in china are gradually becoming popular.

Besides weak hearing or hearing loss, the hearing-impaired people have the same pronunciation organs and sound-changing organs as normal people, and more hearing-impaired people have the ability and conditions of spoken language communication because hearing compensation is obtained by wearing a hearing aid or a cochlear implant. However, the good spoken language expression ability is not achieved at once for the hearing-impaired people, and the people need to be matched with scientific guidance and long-term practice according to the age and the development stage of the individual, gradually promoted and slowly trained. Language is an important ability for humans to express ideas, communicate information, and participate in social interpersonal interactions. Restoring speech expression ability, and communicating through spoken language, and realizing deafness instead of dumb is an important step for hearing-impaired people to be integrated into the society. However, the rehabilitation training of the speech rehabilitation institution is expensive, a lot of families are not capable of supporting the rehabilitation training, or the simple learning is not given up, and a large amount of spoken language correction training is needed in daily life when the spoken language is recovered to a normal level in the later period.

At present, the domestic computer-aided speech audiometry technology is used in scientific research and clinic, but cannot be popularized and applied well. The reason is manifold, from the perspective of a speech recognition and evaluation system, most of domestic speech evaluation and rehabilitation software only emphasizes the accuracy of the system, the audience groups are hearing rehabilitation institutions and trainers, and training equipment for hearing-impaired groups is less; meanwhile, the speech training mode is single, the requirements of different crowds cannot be met, the training mode is backward, and multiple senses and multiple technologies are not integrated.

Disclosure of Invention

In order to overcome the above defects in the prior art, an embodiment of the present invention provides a multi-modal speech rehabilitation training system combining oral muscle training, and the technical problems to be solved by the present invention are: how to better assist the hearing-impaired group to carry out language rehabilitation training by means of computer equipment.

In order to achieve the purpose, the invention provides the following technical scheme: the utility model provides a multi-modal speech rehabilitation training system that musculus oralis training combines, includes human-computer interaction equipment, human-computer interaction equipment includes data acquisition unit, data analysis unit, data contrast unit, display element and database, data acquisition unit and data analysis unit electric connection, data analysis unit and data contrast unit electric connection, data contrast unit is connected with display element and database.

In a preferred embodiment, the display unit comprises a display screen and an operation keyboard which are arranged on the surface of the human-computer interaction device, the display unit is used for practicing content, practicing actions and practicing scores, and the human-computer interaction device is provided with a USB interface.

In a preferred embodiment, the data acquisition unit comprises a sound acquisition module, an image acquisition module and a gas sensing module, wherein the sound acquisition module is a sound pickup embedded in the human-computer interaction equipment, the image acquisition module is a camera arranged at the upper end of the human-computer interaction equipment, and the gas sensing module is a gas pressure sensor arranged at the front end of the human-computer interaction equipment.

In a preferred embodiment, the data analysis unit is used for sorting and analyzing the collected data, and the analysis result is transmitted to the data comparison unit.

In a preferred embodiment, the data comparison unit includes a voice comparison module, an image comparison module, and a pressure comparison module.

In a preferred embodiment, the voice comparison module is configured to compare the audio collected by the sound collection module with a standard audio in a database, and obtain a training score according to a coincidence rate; the image comparison module is used for comparing the mouth shape image acquired by the image acquisition module with a standard mouth shape image in a database and obtaining a training score according to the coincidence rate; the air pressure comparison module is used for comparing the breath rules collected by the air sensing module with standard breath rules in the database and obtaining training scores according to the coincidence rate.

The invention also provides a using method of the multi-modal speech rehabilitation training system combining the oral muscle training, which comprises the following steps,

the method comprises the following steps: the training materials are imported into a database, and the database comprises three degrees of simplicity, complexity and difficulty, and is difficult to train;

step two: the method comprises the following steps that a user selects difficulty during training on a display screen of the human-computer interaction device, pronunciation training is carried out according to display guidance of an interaction interface, and audio, images and exhalation air pressure information of the user are collected by a data collecting unit during the pronunciation training;

step three: the data acquisition unit transmits acquired data information to the data analysis unit, the data analysis unit analyzes the acquired audio, image and air pressure data and transmits an analysis result to the data comparison unit, the data comparison unit compares the training information of the user with the standard pronunciation information in the database and obtains a comprehensive score according to the coincidence rate of the training information of the user and the standard pronunciation information;

step four: if the comprehensive score is lower than the score, the user carries out targeted oral muscle training content;

step five: the user carries out targeted oral muscle training according to the guidance of a display screen on the human-computer interaction equipment, and a data acquisition unit records the targeted oral muscle training process;

step six: and performing secondary pronunciation after the targeted oral muscle training is finished, and comparing and scoring the pronunciation training information and the standard pronunciation information by the data comparison unit until the pronunciation training score exceeds the standard pronunciation information, thereby finishing the training.

In a preferred embodiment, the material for speech training of the hearing-impaired children in the first step includes initial consonants, vowels, single syllables, phrase syllables and tones.

In a preferred embodiment, the targeted oral muscle training in the fourth step comprises pronunciation mouth shape, pronunciation phonetic symbol and gas training, and the training is carried out by selecting the item with lower score according to each score of the first pronunciation training in the third step.

Compared with the prior art, the invention has the technical effects and advantages that:

according to the invention, the pronunciation training and the oral muscle training are performed in a crossed manner by using an audio contrast technology, intelligent recommendation and a modular learning manner, and the learning interest of hearing-impaired children is stimulated by combining with a friendly human-computer interaction interface, so that the accuracy of speech pronunciation is improved, the learning cost of speech training is greatly reduced, and the popularization of speech training is enhanced; the training content is divided into different grades according to the difficulty degree of learning, so that the user can personally select the training content which is more in line with the self level; meanwhile, the pronunciation organ training can be performed according to the difference of different pronunciation organs and the pronunciation score condition of the user, the pronunciation force point is determined, the muscle training of the pronunciation organ is strengthened, and the pronunciation accuracy and the muscle strength of the pronunciation organ are improved.

Drawings

FIG. 1 is a schematic diagram of the system of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, a multi-modal speech rehabilitation training system combining oral muscle training comprises a human-computer interaction device, wherein the human-computer interaction device comprises a data acquisition unit 1, a data analysis unit 2, a data comparison unit 3, a display unit 4 and a database 5, the data acquisition unit 1 is electrically connected with the data analysis unit 2, the data analysis unit 2 is electrically connected with the data comparison unit 3, and the data comparison unit 3 is connected with the display unit 4 and the database 5.

In a preferred embodiment, the display unit 4 comprises a display screen and an operation keyboard which are arranged on the surface of the human-computer interaction device, the display unit 4 is used for practicing content, practicing actions and practicing scores, and the human-computer interaction device is provided with a USB interface.

In a preferred embodiment, the data acquisition unit 1 includes a sound acquisition module 6, an image acquisition module 7 and a gas sensing module 8, the sound acquisition module 6 is a sound pickup device embedded in the human-computer interaction device, the image acquisition module 7 is a camera arranged at the upper end of the human-computer interaction device, and the gas sensing module 8 is a gas pressure sensor arranged at the front end of the human-computer interaction device.

In a preferred embodiment, the data analysis unit 2 is used to collate and analyze the collected data, and the analysis result is transmitted to the data comparison unit 3.

In a preferred embodiment, the data comparison unit 3 comprises a voice comparison module 9, an image comparison module 10 and a pneumatic comparison module 11.

In a preferred embodiment, the voice comparison module 9 is configured to compare the audio collected by the sound collection module 6 with the standard audio in the database 5, and obtain the training score according to the coincidence rate; the image comparison module 10 is used for comparing the mouth shape image acquired by the image acquisition module 7 with the standard mouth shape image in the database 5 and obtaining a training score according to the coincidence rate; the air pressure comparison module 11 is used for comparing the breath law acquired by the air sensing module 8 with the standard breath law in the database 5 and obtaining the training score according to the coincidence rate.

The invention also provides a using method of the multi-modal speech rehabilitation training system combining the oral muscle training, which comprises the following steps,

the method comprises the following steps: the training materials are imported into a database 5, and the database 5 comprises three difficult and easy training degrees of simplicity, complexity and difficulty;

step two: a user selects the difficulty degree during training on a display screen of the human-computer interaction device, pronouncing training is carried out according to the display guidance of an interaction interface, and the audio frequency, the image and the exhalation air pressure information of the user are collected by the data collection unit 1 during the pronouncing training;

step three: the data acquisition unit 1 transmits acquired data information to the data analysis unit 2, the data analysis unit 2 analyzes acquired audio, image and barometric pressure data and transmits an analysis result to the data comparison unit 3, the data comparison unit 3 compares training information of a user with standard pronunciation information in the database 5 and obtains a comprehensive score according to the coincidence rate of the training information of the user and the standard pronunciation information;

step four: if the comprehensive score is lower than 50 points, the user carries out targeted oral muscle training content;

step five: the user carries out targeted oral muscle training according to the guidance of a display screen on the human-computer interaction equipment, and the data acquisition unit 1 records the targeted oral muscle training process;

step six: after the targeted oral muscle training is finished, secondary pronunciation is carried out, and the data comparison unit 3 compares and scores pronunciation training information and standard pronunciation information until the pronunciation training score exceeds 50 to finish the training.

In a preferred embodiment, the material for speech training of the hearing-impaired children in the first step includes initial consonants, vowels, single syllables, phrase syllables and tones.

In a preferred embodiment, the targeted oral muscle training in the fourth step comprises pronunciation mouth shape, pronunciation phonetic symbol and gas training, and the training is carried out by selecting the item with lower score according to each score of the first pronunciation training in the third step.

According to the invention, the pronunciation training and the oral muscle training are performed in a crossed manner by using an audio contrast technology, intelligent recommendation and a modular learning manner, and the learning interest of hearing-impaired children is stimulated by combining with a friendly human-computer interaction interface, so that the accuracy of speech pronunciation is improved, the learning cost of speech training is greatly reduced, and the popularization of speech training is enhanced; the training content is divided into different grades according to the difficulty degree of learning, so that the user can personally select the training content which is more in line with the self level; meanwhile, the pronunciation organ training can be performed according to the difference of different pronunciation organs and the pronunciation score condition of the user, the pronunciation force point is determined, the muscle training of the pronunciation organ is strengthened, and the pronunciation accuracy and the muscle strength of the pronunciation organ are improved.

The points to be finally explained are: first, in the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounted," "connected," and "connected" should be understood broadly, and may be a mechanical connection or an electrical connection, or a communication between two elements, and may be a direct connection, and "upper," "lower," "left," and "right" are only used to indicate a relative positional relationship, and when the absolute position of the object to be described is changed, the relative positional relationship may be changed;

secondly, the method comprises the following steps: in the drawings of the disclosed embodiments of the invention, only the structures related to the disclosed embodiments are referred to, other structures can refer to common designs, and the same embodiment and different embodiments of the invention can be combined with each other without conflict;

and finally: the above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that are within the spirit and principle of the present invention are intended to be included in the scope of the present invention.