阅读说明：本技术 一种基于vr视听觉引导的失语症康复训练系统及训练方法 (Aphasia rehabilitation training system and training method based on VR visual and auditory guidance ) 是由 钱倩 张岩 崔维维 张文婧 王兰 王曼曼 金钧源 于 2021-11-11 设计创作，主要内容包括：本发明公开了一种基于VR视听觉引导的失语症康复训练系统及训练方法,该系统包括传感交互设备、VR设备、外部显示设备和控制平台,传感交互设备用于采集患者的语音和动作信息,VR设备用于引导患者完成训练任务、以及根据患者的语音和动作信息给予反馈,外部显示设备用于显示患者的训练情况。患者在VR环境中通过系统中虚拟人物形象配合语音、文字引导的形式替代了传统失语症康复训练时需要治疗师人工进行语音引导的部分,极大的减轻了治疗师的工作负担,从而提升了康复效率。训练内容设置紧贴Schuell刺激疗法,一方面使训练过程更加系统、规范和专业,另一方面将传统的Schuell刺激训练方法进行了自动化的改造进而提升训练效率。(The invention discloses an aphasia rehabilitation training system and a training method based on VR visual and auditory guidance. The patient has replaced the part that traditional aphasia rehabilitation training needs the therapist manual work to carry out the pronunciation guide through the form of virtual character image cooperation pronunciation, characters guide in the system in the VR environment, very big reduction therapist's work burden to recovered efficiency has been promoted. Training content is set to be tightly attached to Schuell stimulation therapy, so that the training process is more systematic, standard and professional on one hand, and on the other hand, the traditional Schuell stimulation training method is automatically modified to improve the training efficiency.)

1. The VR visual and auditory guidance-based aphasia rehabilitation training system is characterized by comprising sensing interaction equipment, VR equipment, external display equipment and a control platform, wherein the sensing interaction equipment is used for collecting voice and action information of a patient and transmitting the voice and action information to the VR equipment, the VR equipment is used for guiding the patient to complete a training task according to the training task issued by the control platform and giving feedback according to the voice and action information of the patient, the external display equipment is used for displaying the training condition of the patient, and the control platform is used for controlling the VR equipment and the external display equipment.

2. The VR visual and auditory guidance-based aphasia rehabilitation training system according to claim 1, wherein the VR device comprises a task content module, a prompt cue module, and a positive feedback module, the task content module is used for triggering a task according to information of the control platform, the prompt cue module is used for giving a prompt cue when a patient does not respond or responds incorrectly to task content, and the positive feedback module is used for giving positive feedback when the patient responds correctly to task content.

3. The VR visual-auditory guidance-based aphasia rehabilitation training system according to claim 1, wherein the VR device further comprises a preprocessing module for providing an action demonstration, and the control platform generates task preferences according to voice and action information of the patient collected by the sensing interaction device when the preprocessing module runs for triggering task content directly by a doctor or according to the task preferences.

4. The VR visual-auditory guidance-based aphasia rehabilitation training system according to claim 1, further comprising a patient information collection module, wherein the patient information collection module is used for collecting patient personal identity information and patient illness state information and transmitting the patient personal identity information and the patient illness state information to the control platform, and the control platform is used for establishing an account according to the patient personal identity information and giving a corresponding task grade according to the patient illness state information.

5. A training method of the VR visual-auditory-guidance-based aphasia rehabilitation training system according to any of claims 1-4, comprising at least one of the following steps:

s1, training naming ability: the VR equipment displays the category, the use scene or the location information of the target object and provides 3D lens voice description to stimulate a naming reaction, and if naming is correct after prompting, positive feedback is given by VR; if no reaction or error reaction exists, entering a naming prompting step;

s2, training listening comprehension ability: the VR equipment displays a virtual scene according to the listening and understanding task, the virtual character voice guides the patient to point out or select a target object or make a target action, and if the answer is correct, positive feedback is given; if no reaction or wrong answer exists, entering a step of listening and understanding to prompt; the step of listening, understanding and prompting is that the VR equipment completes a task through voice guidance, and if the task is correct, positive feedback is given; if the error occurs, a voice prompt is given;

s3, training character comprehension capacity: converting voice stimulation in an listening and understanding mode into character stimulation, and finishing a character understanding task of pointing out or selecting a target object or making a target action in a VR virtual scene by a voice guide task; if correct, giving positive feedback; if the mistake is made, the voice prompts the patient to think again;

s4, multi-person interactive training: and the VR equipment displays the virtual scene, a plurality of people autonomously select roles or automatically allocate the roles, and the plurality of people jointly complete tasks.

6. The training method of the VR visual-auditory-guidance-based aphasia rehabilitation training system according to claim 5, wherein the naming and prompting step in the step S1 specifically comprises the following steps:

s11, enabling VR equipment to present a radical prompt or a character prompt of the name of the target object, exciting a naming reaction, and if naming is correct after prompting, enabling VR equipment to give positive feedback; if no reaction or error reaction exists, entering the next prompting step;

s12, enabling the VR virtual character to give a statement completion prompt related to the target, and exciting a naming reaction, wherein if the naming is correct after the prompt, the VR gives positive feedback; if no reaction or error reaction exists, entering the next step;

s13, giving a pronunciation mouth shape prompt, a first character or an initial sound prompt to the target first character by the VR virtual character to guide naming reaction, and giving positive feedback by the VR if the naming is correct after the mouth shape prompt; if the mouth shape prompts no reaction or wrong reaction, a first character or an initial sound is prompted, and positive feedback is given for correct naming; if no reaction or error reaction exists, entering the next step;

s14, directly giving correct naming to the target by the VR virtual character, and prompting the patient to repeat 2-3 times by voice; and then switch to the next training goal.

Technical Field

The invention relates to an aphasia rehabilitation training system and a training method based on VR visual and auditory guidance.

Background

The number of people with aphasia in China is increasing due to cerebrovascular accidents or brain trauma and the like. The continuous improvement of the living standard of people ensures that patients and family members pay high attention to and demand on aphasia rehabilitation. However, in China, the number of rehabilitation therapists is far lower than the clinical requirement; in clinical treatment, a one-to-one traditional treatment form is adopted mostly, a large amount of manpower is consumed, and particularly, a therapist is required to perform a large amount of clue prompt description on articles and scenes in a voice guidance part, so that the therapist brings heavy workload; in clinical actual work, training materials are mostly stimulated to patients in a two-dimensional picture and voice mode, and certain gap exists between the training materials and multi-path stimulation treatment advocated by Schuell stimulation therapy, so that the effect of rehabilitation treatment is possibly influenced to a certain degree, and the rehabilitation process of aphasia is prolonged.

Chinese patent CN 107967844 discloses a Chinese aphasia psychological language training method and system, which provides a set of training process and a method for calculating accuracy for aphasia patients. However, the method only describes the traditional process of training the patient by using a two-dimensional picture form, and the picture form guidance cannot replace the induced effect of a three-dimensional virtual object on the patient. On the other hand, pure computer voice guidance often affects a patient's understanding of the guidance and willingness to participate in a conversation because of lack of realism. A VR audiovisual guidance in a virtual reality environment is therefore crucial for a more immersive engagement rehabilitation training of the patient.

The situation that a patient loses speech after stroke is common, different treatment methods are selected after systematic scoring of different injury degrees, and the Schuell stimulation method is a method which is widely applied. The Schuell stimulation method is to apply strong, controlled auditory stimulation to maximally promote reconstruction and recovery of the damaged speech-symbol system by aphasia patients. The Schuell stimulation method firstly has clear understanding on the aspects of occupation, educational experience, life experience, interest and hobbies and the like of patients, repeatedly uses topics which are easily accepted by the patients according to the aphasia degree and the type of the topics, gives corresponding stimulation to the patients through listening, watching, touching and smelling, and ensures that the standard, the mode, the strength and the teaching materials of the stimulation are gradual and cannot be excessively impersonated. When the patient can not make correct reaction, the patient should be given corresponding prompts, such as gesture making, repeated description, pronunciation, speaking, writing, etc., and the patient accompanied with dysarthria should demonstrate the mouth shape and tongue position of pronunciation, and teach correct pronunciation skill. When a treatment plan is formulated, the interest of a patient in receiving stimulation training is improved by selecting rich, vivid and interesting contents, the psychology of the patient is better grasped, a treatment scheme is timely adjusted according to the psychological change and the mental state of the patient on the same day, the habit of active communication of the patient is developed, and the psychology of inferior depression is overcome by exploiting the verbal communication potential of the patient. The Schuell stimulation therapy has wide application and obvious effect, particularly activates nerve cells with low function of the speech central nervous system, increases the number of nerve fibers, strengthens the compensation effect of brain function and has excellent curative effect on the recovery communication ability of aphasia patients in the early stage of recovery of cerebral apoplexy through a stimulation-reaction-feedback loop.

Therefore, there is a need for an aphasia rehabilitation training system based on VR visual-auditory guidance and a training method for implementing Schuell stimulation therapy.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides a system and a method for aphasia rehabilitation training based on VR visual and auditory guidance.

In order to solve the technical problems, the invention provides the following technical scheme:

the invention provides an aphasia rehabilitation training system based on VR (virtual reality) visual and auditory guidance, which comprises sensing interaction equipment, VR equipment, external display equipment and a control platform, wherein the sensing interaction equipment is used for collecting voice and action information of a patient and transmitting the voice and action information to the VR equipment, the VR equipment is used for guiding the patient to complete a training task according to the training task issued by the control platform and giving feedback according to the voice and action information of the patient, the external display equipment is used for displaying the training condition of the patient, and the control platform is used for controlling the VR equipment and the external display equipment.

As a preferred technical solution of the present invention, the VR device includes a task content module, a prompt cue module, and a positive feedback module, wherein the task content module is configured to trigger a task according to information of the control platform, the prompt cue module is configured to give a prompt cue when a patient does not respond to task content or has an incorrect response, and the positive feedback module is configured to give positive feedback when the patient has an correct response to task content.

As a preferred technical solution of the present invention, the VR device further includes a preprocessing module, the preprocessing module is configured to provide an action demonstration, and the control platform generates task preferences according to the voice and action information of the patient collected by the sensing interaction device when the preprocessing module runs, and the task preferences are used for directly triggering task content by a doctor or triggering task content according to the task preferences.

As a preferred technical scheme, the system also comprises a patient information acquisition module, wherein the patient information acquisition module is used for acquiring the personal identity information of the patient and the illness state information of the patient and transmitting the personal identity information and the illness state information to the control platform, and the control platform is used for establishing an account according to the personal identity information of the patient and giving out a corresponding task grade according to the illness state information of the patient.

As a preferred embodiment of the present invention, the present invention provides a training method for a VR visual-auditory guidance-based aphasia rehabilitation training system, including at least one of the following steps:

s1, training naming ability: the VR equipment displays the category, the use scene or the location information of the target object and provides 3D lens voice description to stimulate a naming reaction, and if naming is correct after prompting, positive feedback is given by VR; if no reaction or error reaction exists, entering a naming prompting step;

s2, training listening comprehension ability: the VR equipment displays a virtual scene according to the listening and understanding task, the virtual character voice guides the patient to point out or select a target object or make a target action, and if the answer is correct, positive feedback is given; if no reaction or wrong answer exists, entering a step of listening and understanding to prompt; the step of listening, understanding and prompting is that the VR equipment completes a task through voice guidance, and if the task is correct, positive feedback is given; if the error occurs, a voice prompt is given;

s3, training character comprehension capacity: converting voice stimulation in an listening and understanding mode into character stimulation, and finishing a character understanding task of pointing out or selecting a target object or making a target action in a VR virtual scene by a voice guide task; if correct, giving positive feedback; if the mistake is made, the voice prompts the patient to think again;

s4, multi-person interactive training: and the VR equipment displays the virtual scene, a plurality of people autonomously select roles or automatically allocate the roles, and the plurality of people jointly complete tasks.

As a preferred technical solution of the present invention, the naming prompting step in step S1 specifically includes the following steps:

s11, enabling VR equipment to present a radical prompt or a character prompt of the name of the target object, exciting a naming reaction, and if naming is correct after prompting, enabling VR equipment to give positive feedback; if no reaction or error reaction exists, entering the next prompting step;

s12, enabling the VR virtual character to give a statement completion prompt related to the target, and exciting a naming reaction, wherein if the naming is correct after the prompt, the VR gives positive feedback; if no reaction or error reaction exists, entering the next step;

s13, giving a pronunciation mouth shape prompt, a first character or an initial sound prompt to the target first character by the VR virtual character to guide naming reaction, and giving positive feedback by the VR if the naming is correct after the mouth shape prompt; if the mouth shape prompts no reaction or wrong reaction, a first character or an initial sound is prompted, and positive feedback is given for correct naming; if no reaction or error reaction exists, entering the next step;

s14, directly giving correct naming to the target by the VR virtual character, and prompting the patient to repeat 2-3 times by voice; and then switch to the next training goal.

Compared with the prior art, the invention has the following beneficial effects:

first, the patient has replaced the part that traditional aphasia rehabilitation training needs the therapist manual work to carry out the voice guide through the form of virtual human image cooperation pronunciation, characters guide in the system in the VR environment, very big reduction therapist's work burden to recovered efficiency has been promoted.

Secondly, the patient wears virtual equipment and wears sensing equipment such as gloves or handles to carry out rehabilitation training when necessary, the provided training materials are rich in forms, such as 3D, characters, virtual character images, voice and the like, and the traditional training content limited to two-dimensional picture stimulation is greatly expanded, so that multi-path and multi-channel stimulation is realized, and the training effect is further promoted.

Thirdly, the training content is set to be tightly attached to Schuell stimulation therapy and is included in the Cueing Hierarchy for Anomia naming cue level and the SFA (semantic feature analysis), so that on one hand, the training process is more systematic, standard and professional, and on the other hand, the traditional Schuell stimulation training method is automatically modified to improve the training efficiency.

Fourthly, the training materials simultaneously comprise listening comprehension, expression and character comprehension training, the improvement and interactive promotion of the three language abilities can be comprehensively realized, and the multi-aspect and all-around progress of the language is enhanced.

Drawings

Fig. 1 is a block diagram of a system for rehabilitation training of aphasia based on VR visual and auditory guidance.

Fig. 2 is a block diagram of a VR device in the present invention.

FIG. 3 is a naming task flow diagram in the present invention.

FIG. 4 is a flow chart of an understanding training task in the present invention.

Fig. 5 is a flow chart of training of the life virtual scene in the invention.

FIG. 6 is a flow chart of a multi-person interactive training virtual scene in the present invention.

Detailed Description

The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.

Example 1

In order to achieve the purpose of the present invention, as shown in fig. 1 to fig. 2, this embodiment provides a VR visual and auditory guidance-based aphasia rehabilitation training system, which includes a sensing interaction device, a VR device, an external display device and a control platform, wherein the sensing interaction device is used for collecting voice and motion information of a patient and transmitting the voice and motion information to the VR device, the VR device is used for guiding the patient to complete a training task according to the training task issued by the control platform and giving feedback according to the voice and motion information of the patient, the external display device is used for displaying a training situation of the patient, and the control platform is used for controlling the VR device and the external display device.

Specifically, the VR device comprises a task content module, a prompt cue module and a positive feedback module, wherein the task content module is used for triggering a task according to information of the control platform, the prompt cue module is used for giving a prompt cue when a patient does not respond or reacts incorrectly to task content, and the positive feedback module is used for giving positive feedback when the patient reacts correctly to the task content.

Example 2

In order to further optimize the implementation effect of the present invention, the rest is the same as embodiment 1, except that in this embodiment, the VR device further includes a preprocessing module, the preprocessing module is configured to provide an action demonstration, and the control platform generates task preferences according to the voice and action information of the patient collected by the sensing interaction device when the preprocessing module runs, and the task preferences are used for directly triggering task content by the doctor or triggering task content according to the task preferences.

Example 3

In order to further optimize the implementation effect of the present invention, the rest of the embodiments are the same as those of embodiment 1, except that the present embodiment further includes a patient information acquisition module, the patient information acquisition module is configured to acquire patient personal identity information and patient illness state information and transmit the patient personal identity information and the patient illness state information to the control platform, and the control platform is configured to establish an account according to the patient personal identity information and give a corresponding task level according to the patient illness state information.

Example 4

The invention provides a training method of an aphasia rehabilitation training system based on VR visual and auditory guidance, which comprises the following steps:

s1, training naming ability: the VR equipment displays the category, the use scene or the location information of the target object and provides 3D lens voice description to stimulate a naming reaction, and if naming is correct after prompting, positive feedback is given by VR; if no reaction or error reaction exists, entering a naming prompting step;

s2, training listening comprehension ability: the VR equipment displays a virtual scene according to the listening and understanding task, the virtual character voice guides the patient to point out or select a target object or make a target action, and if the answer is correct, positive feedback is given; if no reaction or wrong answer exists, entering a step of listening and understanding to prompt; the step of listening, understanding and prompting is that the VR equipment completes a task through voice guidance, and if the task is correct, positive feedback is given; if the error occurs, a voice prompt is given;

s3, training character comprehension capacity: converting voice stimulation in an listening and understanding mode into character stimulation, and finishing a character understanding task of pointing out or selecting a target object or making a target action in a VR virtual scene by a voice guide task; if correct, giving positive feedback; if the mistake is made, the voice prompts the patient to think again;

s4, multi-person interactive training: and the VR equipment displays the virtual scene, a plurality of people autonomously select roles or automatically allocate the roles, and the plurality of people jointly complete tasks.

The naming prompting step specifically comprises the following steps:

s11, enabling VR equipment to present a radical prompt or a character prompt of the name of the target object, exciting a naming reaction, and if naming is correct after prompting, enabling VR equipment to give positive feedback; if no reaction or error reaction exists, entering the next prompting step;

s12, enabling the VR virtual character to give a statement completion prompt related to the target, and exciting a naming reaction, wherein if the naming is correct after the prompt, the VR gives positive feedback; if no reaction or error reaction exists, entering the next step;

s13, giving a pronunciation mouth shape prompt, a first character or an initial sound prompt to the target first character by the VR virtual character to guide naming reaction, and giving positive feedback by the VR if the naming is correct after the mouth shape prompt; if the mouth shape prompts no reaction or wrong reaction, a first character or an initial sound is prompted, and positive feedback is given for correct naming; if no reaction or error reaction exists, entering the next step;

s14, directly giving correct naming to the target by the VR virtual character, and prompting the patient to repeat 2-3 times by voice; and then switch to the next training goal.

Example 5

The VR device presents the category of the target, the usage scenario or location information and provides a 3D shot voice description to motivate the naming reaction. This step combines SFA semantic feature analysis based on diffusion activation model as theory to promote naming reaction.

As shown in the naming training flowchart of fig. 3, different categories of training tasks are set, and a certain number of high-frequency words, such as nouns (fruits, animals, vegetables, kitchenware, furniture, vehicles, clothes, places, professions, etc.), verbs, etc., are collected. The naming training prompting step is set to be freely selectable, and before training begins, a therapist selects a targeted training prompting level for the patient so as to be flexibly suitable for various naming disorder patients. And after the training is finished, the VR equipment feeds back the training result and the accuracy, and encourages to improve the training confidence and the enthusiasm. The system sets a training result recording table so that a therapist can conveniently check the training result recording table and adjust a training scheme and difficulty level in time.

3D show fruit stalls (which are a kind of fruit), show apple trees (growing on trees), eat apple actions (can eat); if the naming is correct after prompting, VR gives positive feedback, e.g. answer vs. La, you are true! (vertical thumb or hook may be displayed); if no reaction or error reaction exists, the system enters the next prompting step. Presenting a radical prompt or a character prompt of the name of the target object, and exciting the naming; the step provides character form stimulation, enhances the activation of the vocabulary semantic path and improves the naming reaction. Presenting a Chinese character 'mi' grid character: apple; no increase in response suggests: apple, etc; if the naming is correct after prompting, VR gives positive feedback, e.g. answer vs. La, you are true! (vertical thumb or hook may be displayed); if there is no reaction or error reaction, the system enters the next prompt stage. The avatar gives the target a statement completion prompt (i.e., a voice prompt) related to the target, which triggers a naming reaction. And virtual character prompting: it smells fragrant and tastes sweet; if the naming is correct after prompting, VR gives positive feedback, e.g. answer vs. La, you are true! (vertical thumb or hook may be displayed); if there is no reaction or error reaction, the system goes to the next step. The virtual character gives a pronunciation mouth shape prompt of the first character of the target, a first character or an initial prompt and guides the naming reaction. The virtual character displays the pronunciation mouth shape of 'apple'; if the naming is correct after the mouth shape prompt, VR gives positive feedback, e.g. answer to cheer, you are true! (vertical thumb or hook may be displayed); if the mouth shape prompts no reaction or wrong reaction, a first character (or initial sound) is prompted, and positive feedback is given for correct naming; the virtual character pronounces the first character of the ping; if there is no reaction or error reaction, the system goes to the next step. The virtual character directly gives the target a correct name, and prompts the patient to repeat 2-3 times by voice; and switching to the next training target.

Example 7

As shown in fig. 4, the understanding training task flow chart is used for setting different categories of understanding tasks (noun vocabulary understanding, verb understanding, sentence instruction understanding). Listening comprehension context is VR virtual home living environment (e.g., kitchen, bathroom, bedroom, living room, dining room).

Training the vocabulary listening comprehension of kitchen articles: the virtual scene is a kitchen room, 3 kitchen articles are displayed and comprise target objects (such as bowls, chopsticks and kitchen knives), and equipment gloves or handles are worn; the virtual character voice guides the patient to point out or select the target object (such as please point out or select the bowl); if the answer is correct, give positive feedback, if the answer is right, you are true! If there is no reaction or wrong answer, give a prompt (e.g. round, for meals and snacks) and trigger a reaction. If the answer is correct, giving positive feedback, and entering the next question; if the answer is wrong, the next question is entered after the correct answer is emphasized (the correct target can be independently displayed); after training, VR Speech and text show accuracy, encourage the patient to continue refueling!

The training verb listens to understand: the VR equipment simultaneously displays three action options (such as cutting vegetables, washing faces and brushing teeth), and wears equipment gloves or handles; the VR avatar voice directs the patient to indicate a target action (e.g., please indicate "brush teeth"); if the answer is correct, give positive feedback, if the answer is right, you are true! If no response or wrong answer is given, giving a prompt (such as getting up to clean teeth in the morning) and stimulating the response; if the answer is correct, giving positive feedback, and entering the next question; if the answer is wrong, the next question is entered after the correct answer is emphasized (the correct target can be independently displayed); after training, VR Speech and text show accuracy, encourage the patient to continue refueling!

The training sentences listen to understand: the VR equipment simultaneously displays three items (such as a coat, a wardrobe and a washing machine) and wears equipment gloves or handles; the virtual character voice guides the patient to do target sentence actions (such as please put the coat into a wardrobe); if the answer is correct, give positive feedback, if the answer is right, you are true! If no reaction or wrong answer exists, giving a prompt (such as putting clean clothes in order) and stimulating the reaction; if the answer is correct, giving positive feedback, and entering the next question; if the answer is wrong, the next question is entered after the correct answer is emphasized (the correct target can be independently displayed); after training, VR Speech and text show accuracy, encourage the patient to continue refueling!

Training the character comprehension capacity: setting different categories of word understanding tasks (noun word understanding, verb word understanding and sentence understanding); converting voice stimulation in an listening and understanding mode into character stimulation, finishing a VR voice guide task, such as setting a shopping list or a memo list, and finishing a character understanding task of purchasing a target object or selecting the target object in a VR virtual scene; positive feedback can be given correctly; the patient needs to be prompted by voice to think again by mistake.

Example 8

As shown in a life virtual scene training flow chart of fig. 5, a virtual scene of 3D supermarket shopping is constructed to simulate a real supermarket in daily life. This technique lets the patient wear VR glasses while wearing sensors (gloves) on the needed hand. Tasks requiring patient training are displayed in a virtual reality device. The sensor receives the signal of the actual motion of patient's hand, and the different life scenes that virtual reality provided build the environment of very strong immersive sense for the patient, carry out immersive task training. The sensor forms real-time motion feedback to help the patient effectively complete the target task set in the training scene.

The speech training task in the virtual scene comprises a series of conversation tasks with different difficulty levels, the speech training task is suitable for the requirements of patients with different speech disorder degrees, the tasks are set from easy to difficult, and the computer provides multi-path stimulation for the user through seeing, hearing and touching a plurality of sensory channels.

The therapist may play a virtual role in the virtual scene, such as a time-out carrier, a shopping guide. The virtual character is a realistic 3D character, and the virtual character can simultaneously imitate the lip shape to enhance the character substitution feeling when a therapist conducts voice guidance and conversation.

Taking a living scene of a supermarket as an example, entering an interface of a virtual supermarket, wherein the supermarket contains several types of partitions such as vegetables, fruits, meat, daily necessities and kitchen supplies, and each type has a striking text prompt. When the patient is in the scene, the virtual shopping guide says that the user asks for what can help you, and simultaneously displays corresponding text tasks, and the user is waited at night today to prepare 4 dishes, stir-fried eggs with tomatoes, shredded potatoes, fried meat with hot peppers, salad with fruits and vegetables, and asks for buying corresponding food materials. When the time of the patient in the wrong subarea exceeds 8s, the system feeds back a prompt tone of 'wrong thinking, thinking again', and simultaneously, the interface displays a red graphic fork to prompt the mistake; when the stay time of the patient is more than 8s before reaching the correct subarea, the system feeds back 'cheer-cheer, true stick' and simultaneously displays a green graph to prompt that the hook is correct.

The patient can ask questions of the virtual shopping guide played by the therapist, and the dialog materials for asking questions are:

1) location/route query:

A. asking for where the vegetable zone/fruit zone/meat zone/living goods zone are

The virtual shopping guide responds by voice, and meanwhile, the therapist controls the virtual interface to rotate to the corresponding position;

B. at what position the weighing zone is

The virtual shopping guide responds by voice, and meanwhile, the therapist controls the virtual interface to rotate to the corresponding position;

C. where the cashier is

The virtual shopper answers the voice while the therapist manipulates the virtual interface to the corresponding position.

2) Inquiring specific contents:

A. fruit and vegetable salad needs which food materials

The virtual shopping guide answers by voice and displays corresponding characters at the same time;

B. which egg price is favourable today

The virtual shopper answers in voice.

3) The method comprises the following steps:

A. you good! I need a large shopping bag/a cart;

B. you good! My mobile phone is out of power, asks where to borrow the charger baby

C. You good! Asking where to have restroom

The difficulty level increases and some specific tasks can be set. For example, the phone task: when shopping, the patient can listen to the telephone call from a friend and ask the friend to buy the required things. For example, when a clock with friend telephone content stops, 3 batteries and 5 batteries are needed; two red colored ball point pens and 1 notebook were also required. After the phone is over, the patient starts to buy something and the dialog material that the patient can ask is similar to the above. And meanwhile, the cognitive comprehension ability and the memory ability of the patient are exercised. At the end of the training, the system will record the number of times the patient completed the task, the elapsed time, and the difficulty of the task. Therefore, the therapist can adjust the difficulty of the next task according to each training result.

In the embodiment, the virtual reality display equipment is used for immersing the patient in a vivid environment, and daily conversation communication is simulated more truly, so that the actual communication is quickly adapted and integrated; compared with the traditional treatment method, the VR design training can better improve the speech cognitive comprehension ability and the expression ability of aphasia patients, thereby avoiding social embarrassment; the task-oriented and game-oriented training form can relieve the dryness of the training, increase the willingness of the patient to actively participate and improve the compliance of the patient. The pair of the glasses with the VR function can be used for different life scenes, such as a family environment, a supermarket, a restaurant, a hospital, a bank and the like, so that the patients can participate in daily life more, the design is applied to adult aphasia patients, and the treatment of emotional emotion after cerebral apoplexy and cerebral trauma can be helped, so that the self-respect and the self-confidence of the adult aphasia patients are enhanced, and the psychological pressure is relieved.

Example 9

As shown in fig. 6, a flow chart of a multi-person interactive training virtual scene is used to construct a 3D virtual scene to simulate a real supermarket in daily life. This technique lets the patient wear VR glasses while wearing sensors (gloves) on the needed hand. Tasks requiring patient training are displayed in a virtual reality device. The sensor receives a signal of the actual movement of the patient's hand. Different life scenes provided by the virtual reality create a strong immersive environment for the patient to carry out immersive task training. The sensor forms real-time motion feedback to help the patient effectively complete the target task set in the training scene.

In training of an aphasia rehabilitation system for VR multi-person group training, a therapist needs to perform pre-activity patient matching and topic preparation work according to the patient level. The patient can select topics according to self intention or select according to level difficulty prompts (difficulty level five-star level), and the optimal choice is bidirectional choice between a therapist and the patient in task adaptation.

Firstly, the method comprises the following steps: topic task: taking the 'fresh flowers' theme in the flower shop scene as an example, the team members and therapists (doctors, family members and friends) enter the virtual flower shop, and the flower shop contains different types of fresh flowers and related furnishings. The panelists may visit the florist after entering the scene. The therapist then guides the patient to develop the theme. The reference flow is as follows:

1. welcome team members to participate in this topic- "flower shop party".

2. A simple self-introduction is made.

3. Determining the role: guests (active participants), store owners (host), team members themselves, or therapists. 4. Subject related question-what flowers you like

5. Share oneself with experiences related to the topic- "flowers".

6. Scenario, display related discussion-how this flower shop is, you want to buy those flowers here.

7. And (6) ending.

8. System-self-evaluation feedback. The training time of 4-6 panelists is 1-1.5 hours.

II, secondly: in the subject task group training, the outdoor barbecue subject is taken as an example.

1. The therapist and the panelists meet in a virtual "conference room". And performs task notification. Text posters, audio prompts are as follows: the panelists were set to four. Each of the team members configures a virtual cell phone. The cost of the barbecue activity is set to be 500 yuan, and the expense includes the traffic cost, food materials, cookers and the like. Virtual active time 3 hours. Under the condition of not exceeding the expenditure, 20 bunches of beef, 8 bunches of chicken wings and 8 bunches of test vegetable rolls (each bunch contains three kinds of vegetables) are roasted within 3 hours, and the smooth customs clearance is calculated. "meeting room" travel discussion. The therapist assists in the following discussion: explicit barbeque place-four place options (including traffic, charging information, etc.); defining supermarket information-two supermarket position traffic information; defining a travel mode-route; making a shopping list clear; the task of each person is claimed (explicit teams performing the task together, individuals performing the task individually, role responsibilities division: e.g. explicit how finances are assigned and kept). 3. And (4) traffic travel: from a conference room to a supermarket (each travel mode corresponds to virtual time and cost); a subway station; a bus station: communicating with the driver (setting the driver to answer); a taxi: communicating with a driver; the bicycle is shared. 4. Supermarket shopping-finding food materials: performing task execution according to the purchase list (in combination with the supermarket shopping theme in scene training); selecting food materials and tools in vegetable areas, meats, snacks, beverages, tableware, fuels, seasonings, semi-finished quick-frozen strings and the like. 5. Supermarket shopping-supermarket settlement: and (combined with scene training) group personnel convergence, telephone communication, scene switching and staff consultation. 6. From supermarket to barbecue site: (each travel mode corresponds to virtual time and cost) subway stations; a bus station: communicating with a driver; a taxi: communicate with the driver (traffic jam); the bicycle is shared. 7. Preparing for barbecue; task allocation-tools, food seasoning/semi-finished product (different food material roasting time is different, and the duration of heating is controlled). 8. Completion of the task/task failure. 9. System-self-evaluation feedback.

The embodiment reduces the threshold of preparing a physical environment and props for traditional multi-person training through virtual scenes and props, and reduces the workload of therapists and the preparation cost. Using the virtual reality display device, the patient may be immersed in a realistic environment. The patient simulates a real scene and a character relation through a group interaction mode of a plurality of people, reduces the communication pressure and relieves the social embarrassment. Thereby better transiting to the actual communication and rebuilding or improving the social family relationship. The computer port login mode of the equipment is beneficial to remote rehabilitation and family rehabilitation. Many people task direction formula training promotes recovered experience, improves training interest, increases patient compliance. The participation of family members can be improved.

Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.