阅读说明：本技术 一种面向手势控制的人机交互系统和方法 (Gesture control-oriented human-computer interaction system and method ) 是由 方鹏 王迎英 汤熙 田岚 李向新 郑悦 张浩诗 李光林 于 2021-06-28 设计创作，主要内容包括：本发明公开了一种面向手势控制的人机交互系统,包括：腕带本体设置有微针阵列电极；在机械手的每个手指位置均安装压力传感器,用于检测机械手与物体之间的接触信号,经过无线通信方式将所检测到的接触信号传输至所述腕带本体内设置的数据处理与发送模块；数据处理与发送模块,与腕带本体内设置的微针阵列电极电连接,用于对所接收到的信号进行分析并判断感觉信息所属的类型,最后利用判断结果来编码电刺激参数,并将编码好的电流脉冲通过腕带本体上的微针阵列电极施加于使用者手部来刺激诱发和传导手部感觉信息。能够实现手势控制过程的真实自然的感觉反馈功能,将操作者的手从传统的力反馈数据手套中解放出来,可以进行更加灵活的手势操作。(The invention discloses a gesture control-oriented man-machine interaction system, which comprises: the wrist strap body is provided with a microneedle array electrode; a pressure sensor is arranged at each finger position of the manipulator and used for detecting a contact signal between the manipulator and an object, and the detected contact signal is transmitted to a data processing and sending module arranged in the wrist strap body in a wireless communication mode; and the data processing and sending module is electrically connected with the micro-needle array electrodes arranged in the wrist strap body and used for analyzing the received signals and judging the type of the feeling information, finally, the judgment result is used for coding the electrical stimulation parameters, and the coded current pulses are applied to the hands of the user through the micro-needle array electrodes on the wrist strap body to stimulate, induce and conduct the hand feeling information. The real and natural feeling feedback function of the gesture control process can be realized, the hand of an operator is liberated from the traditional force feedback data glove, and more flexible gesture operation can be carried out.)

1. A man-machine interaction system facing gesture control is characterized in that: comprises that

The wrist band comprises a wrist band body, wherein a micro-needle array electrode is arranged in the wrist band body;

a pressure sensor is arranged at each finger position of the manipulator and used for detecting a contact signal between the manipulator and an object and transmitting the detected contact signal to a data processing and sending module arranged in the wrist strap body in a wireless communication mode;

and the data processing and sending module is electrically connected with the micro-needle array electrodes arranged in the wrist strap body and used for analyzing the received signals and judging the type of the feeling information, finally, the judgment result is used for coding the electrical stimulation parameters, and the coded current pulses are applied to the hands of the user through the micro-needle array electrodes on the wrist strap body to stimulate, induce and conduct the hand feeling information.

2. The human-computer interaction system facing gesture control as claimed in claim 1, wherein: the wrist strap body is divided into three layers, and the outermost layer is a non-woven fabric layer; the middle layer is a basal layer, and a microneedle array electrode is arranged on the middle basal layer; the innermost layer in direct contact with the skin is a silicone layer that has been pre-perforated at the location corresponding to the microneedles in the basal layer.

3. The human-computer interaction system facing gesture control as claimed in claim 1, wherein: the wrist strap body is provided with micro-needle array electrodes which correspond to each finger position of the manipulator and can induce real finger feeling respectively.

4. The human-computer interaction system facing gesture control according to claim 3, wherein: the microneedle array electrode includes an electrode substrate and a microneedle array formed on the electrode substrate.

5. The human-computer interaction system facing gesture control according to claim 4, wherein: the electrodes comprise gel wet electrodes, micro-needle array electrodes and various electrodes and combinations thereof.

6. The human-computer interaction system facing gesture control as claimed in claim 1, wherein: the pressure sensor comprises at least one of a pressure sensitive sensor and a piezoelectric sensor.

7. The human-computer interaction system facing gesture control as claimed in claim 1, wherein: the data processing and sending module comprises a signal processing module and a low-power-consumption Bluetooth module connected with the signal processing module, the signal processing module receives a sensor output signal value transmitted from the pressure sensor on the manipulator through the low-power-consumption Bluetooth module, and different threshold values are set to judge the contact state.

8. The human-computer interaction system facing gesture control according to claim 7, wherein: the contact state includes the type of contact feeling, the location of contact, and the degree of closeness of contact.

9. The human-computer interaction system facing gesture control according to claim 7, wherein: the signal processing module firstly carries out filtering and denoising pretreatment on signals transmitted by the pressure sensor to obtain clear pressure signals, then carries out windowing, feature extraction and pattern recognition treatment on the signals, compares a calculated result with a preset threshold value of the previous part, finally judges a recognition result, determines the type of sensory information, compares the judged sensory intensity with the preset threshold value to grade the sensory intensity, codes stimulation parameters and stimulation positions according to the recognition and grading results, and then transmits electric signals to corresponding electrodes to apply electric stimulation to skin and nerves below the electrodes, so that the senses of different intensities at different positions of a hand are induced.

10. A man-machine interaction method facing gesture control is characterized in that: the method comprises the following steps:

providing a gesture control-oriented human-computer interaction system; the man-machine interaction system facing gesture control comprises: the wrist strap comprises a wrist strap body, wherein microneedle array electrodes are arranged in the wrist strap body, a pressure sensor is arranged at each finger position of a manipulator, and a data processing and sending module is electrically connected with the microneedle array electrodes arranged in the wrist strap body;

the pressure sensor is used for detecting a contact signal between the manipulator and an object, and the detected contact signal is transmitted to a data processing and sending module arranged in the wrist strap body in a wireless communication mode;

the received signals are analyzed through the data processing and sending module, the type of the sensory information is judged, finally, the electrical stimulation parameters are coded through the judgment result, and the coded current pulse is applied to the hand of a user through the micro-needle array electrode on the wrist strap body to stimulate, induce and conduct the hand sensory information.

Technical Field

The invention relates to the technical field of wearable equipment, in particular to a gesture control-oriented human-computer interaction system and method.

Background

The gesture control method aims at the problems that an intuitive sense feedback function is lacked in the existing gesture control method, and a user cannot really sense the gesture control effect, so that accurate control cannot be achieved.

The gesture control means that the hand of a person does not need to directly contact the machine, and the machine can be controlled to complete corresponding actions only by making corresponding posture changes in the air. Gesture control is a brand new interaction mode developed in recent years, and is different from common interaction modes such as key pressing and voice, and the gesture control is easier to master and use. The gesture control has important application value in many fields, can realize remote control especially under the hazardous environment, need not the operator and attend the scene, can guarantee operator's personal safety.

The basic elements of the sensing device, the processing chip and the algorithm can not be separated from the gesture control. The method comprises the steps of predefining a control instruction library comprising multiple functions, then acquiring gesture information in real time, identifying and judging gesture actions of an operator through various gesture identification methods, and finally corresponding action judgment results to corresponding control commands in the instruction library so as to realize gesture control. At present, there are several methods for realizing gesture recognition:

the flight time technology. This technique requires equipping a 3D camera module with transmitting and receiving pulsed light. Firstly, the camera module emits pulse light, and the time for the fingers at different distances to receive the light irradiation is different, so that the time for the light to return to the receiving module is also different. According to different return times, the processing chip can calculate the specific positions of different fingers, so that the gesture is recognized.

② structured light technique. Firstly, structured light is projected to the surface of a human body in front by a laser emitter, and then a structured light pattern reflected by the human body is received by an infrared sensor. The processing chip calculates the space information of the human body according to the position and the deformation degree of the received pattern, and then carries out depth calculation by combining a certain algorithm, so that the gesture can be recognized.

And thirdly, millimeter wave radar. The basic principle is similar to time-of-flight techniques, except that the medium used for measurement is changed from light to electromagnetic waves. Firstly, radio waves are transmitted, then echoes are received, and the processing chip calculates target position data in real time according to the transmitting-receiving time difference. And comparing the positions of the fingers in different time periods and comparing the positions with built-in data so as to recognize the gesture.

Data gloves. The data glove is a multi-mode virtual reality hardware, and can perform actions such as grabbing, moving, rotating and the like of objects in a virtual or real scene through software programming. The data glove is provided with a bending sensor, can detect finger bending, and utilizes a magnetic positioning sensor to position the hand in a three-dimensional space. Such a data glove that combines finger curvature and spatial positioning is called a "real glove," and can provide a real natural three-dimensional interaction means for the user.

Generally, the current technology can accurately recognize user gestures in real time and transmit the gestures to a virtual or real control system to realize control application. However, an effective feedback means is not available, and the control effect (such as contact information of a virtual hand or a real robot hand) cannot be fed back to the operator.

The existing gesture control technology mainly realizes a motion control function, realizes remote control through gesture recognition, but lacks a sense feedback function, so that the control effect is poor and the use experience is poor. Only one-way information transmission from the human to the machine is realized, feedback from the machine to the human is lacked, two-way information transmission is not realized, and real man-machine interaction is not realized. For example, when a user remotely controls a manipulator to grip an object through a gesture control technology, due to the lack of an intuitive sensory information feedback function, the user cannot acquire various information as a real human hand during gripping or can only know the basic gripping condition through indirect feedback means such as vision, cannot determine whether to increase or decrease the gripping force by judging whether the object is gripped, cannot sense information such as the material, shape and temperature of the gripped object, and cannot timely respond to external stimulation of the environment where the manipulator is located.

As described above, there are indirect feedback techniques, such as providing light or sound to indicate the strength of the grip when the manipulator grips an object. However, such non-intuitive feedback information cannot be intuitively perceived by the user, and the feedback information is limited and is also easily limited by various environments. In addition, people also develop a force feedback glove technology, devices such as a vibrating reed and the like are added in the data glove, and the method of mechanically stimulating the fingers of a user is used for transmitting sensory information, so that the feedback effect is achieved.

Disclosure of Invention

The invention aims to provide a gesture control-oriented human-computer interaction system to solve the problems in the background technology.

In order to achieve the above object, an embodiment of the present invention provides a gesture control-oriented human-computer interaction system, including:

the wrist band comprises a wrist band body, wherein a micro-needle array electrode is arranged in the wrist band body;

a pressure sensor is arranged at each finger position of the manipulator and used for detecting a contact signal between the manipulator and an object and transmitting the detected contact signal to a data processing and sending module arranged in the wrist strap body in a wireless communication mode;

and the data processing and sending module is electrically connected with the micro-needle array electrodes arranged in the wrist strap body and used for analyzing the received signals and judging the type of the feeling information, finally, the judgment result is used for coding the electrical stimulation parameters, and the coded current pulses are applied to the hands of the user through the micro-needle array electrodes on the wrist strap body to stimulate, induce and conduct the hand feeling information.

Preferably, the wrist strap body is divided into three layers, and the outermost layer is a non-woven fabric layer; the middle layer is a basal layer, and a microneedle array electrode is arranged on the middle basal layer; the innermost layer in direct contact with the skin is a silicone layer that has been pre-perforated at the location corresponding to the microneedles in the basal layer.

Preferably, a micro-needle array electrode corresponding to each finger position of the manipulator is arranged on the wrist band body, and the micro-needle array electrode can induce real finger feeling.

Preferably, the microneedle array electrode includes an electrode substrate and a microneedle array formed on the electrode substrate.

Preferably, the electrodes include gel wet electrodes and microneedle array electrodes of various types and combinations thereof.

Preferably, the pressure sensor comprises at least one of a pressure sensitive sensor, a piezoelectric sensor.

Preferably, the data processing and sending module includes a signal processing module and a bluetooth low energy module connected thereto, and the signal processing module receives a sensor output signal value transmitted from the pressure sensor on the manipulator through the bluetooth low energy module, and sets different thresholds to determine the contact state.

Preferably, the contact state includes a type of contact feeling, a position of contact, and a degree of closeness of contact.

Preferably, the signal processing module firstly performs filtering and denoising preprocessing on a signal transmitted by the pressure sensor to obtain a clear pressure signal, then performs windowing, feature extraction and pattern recognition processing on the signal, compares a calculated result with a threshold set in the previous part, finally judges a recognition result, determines the type of sensory information, compares the judged sensory intensity with the set threshold to grade the sensory intensity, encodes stimulation parameters and stimulation positions according to the recognition and grading results, and then transmits an electric signal to a corresponding electrode to apply electric stimulation to skin and nerves below the electrode, so as to induce the sensation of different strengths at different positions of the hand.

The embodiment of the invention also provides a man-machine interaction method facing gesture control, which comprises the following steps:

providing a gesture control-oriented human-computer interaction system; the man-machine interaction system facing gesture control comprises: the wrist strap comprises a wrist strap body, wherein microneedle array electrodes are arranged in the wrist strap body, a pressure sensor is arranged at each finger position of a manipulator, and a data processing and sending module is electrically connected with the microneedle array electrodes arranged in the wrist strap body;

the pressure sensor is used for detecting a contact signal between the manipulator and an object, and the detected contact signal is transmitted to a data processing and sending module arranged in the wrist strap body in a wireless communication mode;

the received signals are analyzed through the data processing and sending module, the type of the sensory information is judged, finally, the electrical stimulation parameters are coded through the judgment result, and the coded current pulse is applied to the hand of a user through the micro-needle array electrode on the wrist strap body to stimulate, induce and conduct the hand sensory information.

The method of the invention has the following advantages:

(1) the invention can realize the real and natural feeling feedback function of the gesture control process, has more immersion feeling and can experience the control process more intuitively compared with the control process without feeling feedback or indirect feedback depending on vision and the like, and the hands of an operator are liberated from the traditional force feedback data gloves, so that more flexible gesture operation can be carried out, the invention is beneficial to optimizing the use experience and realizing more flexible and fine control.

(2) The method provided by the invention has small calculation amount and is easy to realize by a single chip microcomputer, related calculation methods can be integrated into wearable equipment, and compared with force feedback gloves, the hardware part of the method is mainly made of integrated circuits without moving mechanical elements, so that the power consumption is lower and the response is faster.

(3) Through the natural feeling feedback function, the user can make autonomous judgment after intuitively knowing the control result, continuously revise own gesture and gripping power, and implement more accurate control.

(4) The microneedle array electrode used in the invention is made of a flexible substrate and is arranged on the elastic wrist band. Ensures good adhesion to the skin, and a lower and more stable electrode-skin interface impedance and greater resistance to interference during extended wear and use than wet and other types of dry electrodes, allowing the operator to wear the wristband for extended periods of time.

(5) The manufacturing process of the microneedle array electrode, the elastic bandage and the like adopted by the wrist strap is simple, and the wrist strap is suitable for machine assembly line production.

Drawings

Fig. 1 is a schematic diagram of a sensor space structure of a gesture control-oriented human-computer interaction system provided by the invention.

Fig. 2 is a diagram of a corresponding relationship between an electrical stimulation position and a contact position of the gesture control-oriented human-computer interaction system provided by the invention.

Fig. 3 is a structural diagram of a wrist band body of the gesture control oriented human-computer interaction system in fig. 2.

FIG. 4 is a schematic diagram of a gesture control oriented human-computer interaction system provided by the invention.

FIG. 5 is a flowchart of a gesture control oriented human-computer interaction method provided by the present invention.

Detailed Description

The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention. In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the invention without any inventive step, are within the scope of protection of the invention.

The embodiment of the invention provides a gesture control-oriented man-machine interaction system, which comprises: the wrist strap comprises a wrist strap body 203, a micro-needle array electrode 204 arranged in the wrist strap body 203, a pressure sensor 202 arranged at each finger position of the manipulator 201, and a data processing and sending module 205 electrically connected with the micro-needle array electrode 204 arranged in the wrist strap body 203. A microneedle array electrode 204 is arranged in the wrist band body 203; the pressure sensor 202 is configured to detect a contact signal between the manipulator 201 and an object, and transmit the detected contact signal to a data processing and transmitting module 205 disposed in the wristband body 203 in a wireless communication manner; and the micro-needle array electrodes 204 are electrically connected with the micro-needle array electrodes 204 arranged in the wrist band body 203, and are used for analyzing the received signals and judging the type of the feeling information, finally, the judgment result is used for coding the electrical stimulation parameters, and the coded current pulses are applied to the hands of the user through the micro-needle array electrodes 204 on the wrist band body 203 to stimulate, induce and conduct the hand feeling information.

When the manipulator 201 or the virtual hand contacts an object, for the manipulator 201, electrical stimulation parameters are encoded according to sensory information received from the finger surface sensor (for the virtual hand, sensory information such as whether the contact is made or not and the contact tightness is represented by deformation of a contact surface when the virtual hand grips the object through a program simulation hand), and then surface electrical stimulation is performed on the wrist of a wearer to realize transmission of the sensory information to the brain, so that an operator can adjust the gripping posture and gripping force of the operator according to the sensed contact information, the operator can interact with the virtual world in a more direct, natural and effective mode, interactivity and immersion are greatly enhanced, and human-computer interaction experience is improved. In addition, a universal human-computer interaction mode is provided for an operator, and the method is particularly suitable for a virtual reality system which needs a multi-degree-of-freedom hand model to perform complex operation on a virtual object.

The embodiment of the invention provides a gesture control-oriented man-machine interaction system which comprises two parts, wherein the first part is used for acquiring sensation information and acquiring, transmitting and processing a plurality of modules through signals, a pressure sensor 202 (virtual hand simulation is used) is arranged on a manipulator 201 and used for detecting the contact state or the holding state of the manipulator 201 and an object in the gripping process and transmitting the acquired signals to a sensation wrist strap body 203, a data processing module in the wrist strap analyzes the received signals and judges the type of the sensation information, and finally, the judgment result is used for coding electrical stimulation parameters and inducing the real sensation of the hand of a user; the second part is sensory information feedback, electrical stimulation parameters are coded and set according to the result of the sensory information analysis obtained in the first part, and coded current pulses are applied to the hands of the user through the electrodes on the wrist band to stimulate and induce and conduct hand sensory information.

(1) Sensory information acquisition

For the manipulator 201, the space structure of the sensor is schematically shown in fig. 1, and 101 is the finger surface of the manipulator 201. The layer 102 is the pressure sensor 202, which can be made into a thickness of tens of microns according to the requirement, and does not influence the daily use of the manipulator 201. 103 is a protective layer covering the surface of the pressure sensor 202, and the pressure sensor 202 can be protected by a plastic package process, so that the service life is prolonged. The pressure sensor 202 may be installed at each finger position of the robot 201 according to the use requirement to recognize the state of the object touched at different positions. If the interactive control is the virtual hand, the contact state is judged in the software.

In the aspect of signal processing, the pressure sensor 202 with high sensitivity is used, the output signal value of the sensor is monitored in real time, different threshold values are set to judge the contact state (including the type of contact feeling, the position of contact and the contact tightness: the contact state is divided into different grades of light, medium and heavy, each grade of each feeling type is set as a feeling category), and the data collected by the sensor is transmitted to the next part of the wrist strap for feeling feedback.

(2) Sensory information feedback

As described above, after comparing and classifying the detected pressure with the set threshold value for encoding the electrical stimulation parameter and performing sensory feedback through the wrist band, the present invention induces a sensation using the transdermal electrical stimulation method of the wrist to realize the sensory feedback.

Electrical stimulation induces sensory information afferent: a short-duration (50ms-200ms) pulse of current is applied to the skin surface through the microneedle electrodes. According to different positions of the contact generated by the manipulator 201, an electrode at a wrist skin position right above nerves (mainly including a median nerve, an ulnar nerve, a radial nerve, branches thereof and the like) for controlling different finger activities in the wrist band is selected as a target stimulation electrode, corresponding nerves are activated, sensory information is induced, and the induced sensory information is transmitted to the brain, so that a user generates visual perception of the contacted object, the force for grasping the object is adjusted, and the like, and sensory feedback is realized. The correspondence between the positions of the electrostimulation electrodes and the positions of the fingers of the manipulator 201 in contact with the object is shown in fig. 2.

In the figure, flexible pressure sensors 202 are respectively installed on the fingers of the manipulator 201 at positions corresponding to the numbers 1, 2, 3, 4 and 5, and microneedle array electrodes 204 capable of respectively inducing the corresponding real finger sensations are installed on the wrist band at positions corresponding to the numbers 1, 2, 3, 4 and 5. The micro-needle array is manufactured by pressing, stretching and breaking curable magnetic liquid drops on a flexible substrate under the combined action of magnetic field force, surface tension and gravity, and a conductive metal film with biological safety is sprayed on the surface of the micro-needle, so that the micro-needle array electrode 204 is manufactured. The manufactured electrodes are mounted on a stretchable flexible wristband having biosafety, so that operators with different arm thicknesses can use the electrodes.

The signal processing module CPU206 and the low-power consumption Bluetooth module BLE207 are packaged on the outer surface of the man-machine interaction system facing gesture control, and the low-power consumption Bluetooth module 207 can receive sensory information transmitted from the flexible pressure sensor 202 of the manipulator 201 and can improve the cruising ability of the feedback system; in the signal processing module 206, the signal transmitted by the pressure sensor 202 is first preprocessed by filtering, denoising, etc. to obtain a clear pressure signal, then the signal is processed by windowing, feature extraction (time domain features related to peak value and energy), pattern recognition, etc., the calculated result is compared with a threshold value set in the last part, finally the recognition result is judged, the type of the sensory information is determined, the judged sensory intensity is compared with the set threshold value to grade the sensory intensity, the stimulation parameters (frequency, pulse width and amplitude) and the stimulation position are encoded according to the recognition and grading result, and then the electrical signal is transmitted to the corresponding electrode through a lead to apply electrical stimulation to the skin, nerves, etc. below the electrode, thereby inducing the sensation of different intensities at different positions of the hand.

In addition, because parameters such as pulse width, amplitude, frequency and the like of the current of the percutaneous electrical stimulation are adjustable, various receptors and nerve endings in the skin can be stimulated to different degrees, so that different types of feelings such as light touch, pressing, buzzing, vibration, numbness, pain and the like can be induced. Therefore, a series of sensory information such as pressure, temperature and the like on the contact surface between the manipulator 201 and the gripped object can be transmitted to the skin, nerves and brain through the electrical stimulation electrode after signal processing to a great extent, so that an operator can intuitively feel the gripping feeling of the manipulator 201, and the gripping posture, force and the like can be adjusted according to the type and strength of the sensed feeling, thereby realizing closed-loop control based on natural sensory feedback and improving the performance of controlling the manipulator 201 to complete fine motion tasks.

The structure of the sensing wrist band body 203 is shown in fig. 3, and the whole body is divided into three layers, wherein the outermost layer is a non-woven fabric layer and is connected with the elastic wrist band; the middle layer is a substrate layer, the microneedle array electrode 204 is arranged on the middle layer and is externally connected with the signal processing module 206 and the Bluetooth communication module; the innermost layer in direct contact with the skin is a silicone layer, which has been perforated beforehand in the base layer in a position corresponding to the electrode, in preparation for the contact of the electrode with the skin.

As shown in fig. 4, the working principle of the present invention is as follows:

the operator uses gestures to control the manipulator 201 or virtual hand;

a contact signal such as pressure is detected by a flexible pressure sensor 202 mounted on the robot 201;

transmitting the detected contact signals such as pressure and the like to a man-machine interaction system facing gesture control in a wireless communication mode;

after signal processing and pattern recognition, the wrist band is used for coding electric stimulation parameters corresponding to the sensation, and corresponding stimulation current is transmitted to the microneedle array electrode 204 at a specific position to implement electric stimulation;

inducing the real hand feeling of the operator and transmitting it to the brain;

the operator can adjust the gesture and the grip strength according to the sensed feeling information.

The system is a closed-loop control system, and compared with the traditional gesture control, an operator can control the manipulator 201 to make more gestures due to no constraint of gloves, so that flexible control of multiple modes is realized; and the natural feeling feedback function is added, so that the operator can continuously modify the gesture and the grip strength according to the feeling of the operator, and more accurate control of the manipulator 201 is realized.

(1) The present invention includes a human-machine interaction wristband for implementing a sensory feedback function controlled by a manipulator 201 or virtual hand. The contact information is fed back to the real human hand through the interactive wrist band according to the degree of contact with the object when the manipulator 201 or the virtual hand grips the object (or the virtual object).

(2) The electrical stimulation based sensory wristband includes an electrode layer, a protective layer, a data processing and transmission module 205.

(3) A pressure sensor 202 is mounted on the manipulator 201 to detect the contact between the manipulator 201 and the object, and the pressure sensor 202 includes, but is not limited to, a pressure-sensitive sensor, a piezoelectric sensor, and the like.

(4) The processing and sending module is made of a flexible circuit board, packaged together and fixed at the outer surface of the wrist strap.

(5) Electrodes for electrical stimulation include, but are not limited to, various types of electrodes such as gel wet electrodes and microneedle array electrodes 204, and combinations thereof.

(6) Electrical stimulation is performed through the electrode layer.

(7) The wear is prevented by the protective layer.

(8) The signal and encoded stimulation parameters are processed by a data processing and transmitting module and the stimulation current is transmitted to the electrode layer.

(9) The layers are secured by adhesives including, but not limited to, hot melt adhesives, double sided tapes, and the like, which do not interfere with the transmission of pressure signals and electrical signals after the adhesive is dried.

(10) The electric stimulation electrode bracelet can be fixed in on the elasticity bandage in order to satisfy the operator of wrist thickness difference and use.

(11) The man-machine interaction device can be taken down and moved to another arm for use.

The invention is proved to be feasible through demonstration and preliminary experiments. In one aspect, the method for inducing and transmitting hand sensory information by surface electrical stimulation provided by the invention has been completed by verification experiments on more than 5 subjects, determines appropriate stimulation parameters, and induces multiple senses of different finger and palm areas of the subjects. On the other hand, the micro-needle array electrode 204 provided by the invention is designed and manufactured, and experimental verification of electrical stimulation induced sensory information is completed on the wrist of a human body, and the hand feeling of a subject can be induced only by using a tiny current.

In addition, the data analysis method provided by the invention has the advantages of small calculated amount, easy integration in a system, low additional adding cost and easy technical realization.

The man-machine interaction method based on the sensory feedback provided by the invention is not limited to the control of the mechanical arm 201 or the virtual hand, and can also be used for the sensory feedback in various control scenes such as life, work, experiments and the like. If the temperature is controlled by gestures, sensory feedback is carried out according to different temperatures; controlling the aircraft by using gestures, and performing sensory feedback and the like according to the speed of the aircraft; in a biological experiment, when test tubes made of different materials and filled with different solvents and the like are taken and transferred, sensory feedback and the like are carried out according to the positions, the deformation and the like of the test tubes; in game training, the movement of the virtual object is controlled through gestures to perform feedback according to the contact feeling simulated by a program; and various everyday situations where gesture control is required, the sensory feedback method proposed in the present invention can be used.

The wrist band material in the invention can be replaced by various flexible materials which do not have danger to the skin of a human body, such as flexible silica gel materials and the like, and the shape of the wrist band can be designed according to the requirements of users. The size, shape, number, arrangement and the like of the stimulating electrodes in the wrist band can be adjusted according to the requirements of users and different wrist nerve distributions of different users.

The number, shape, height, diameter, arrangement and the like of the microneedles of the microneedle array electrode 204 can be adjusted according to actual needs.

The electrode includes but is not limited to a wet electrode or a contact dry electrode and a microneedle array electrode 204, the microneedle array electrode 204 is a special dry electrode, mainly uses a curable magnetic liquid drop as a raw material, and forms a microneedle array on a flexible substrate through pressing, stretching and breaking by a lever under the combined action of magnetic field force, surface tension and gravity, and then a conductive film is plated on the surface of the microneedle array, wherein the conductive film includes but is not limited to gold, silver, titanium, silver chloride and other materials. The flexible microneedle array electrode 204 has the following features:

1. the volume is small and the thickness is thin;

2. the processing steps are simple, the reproducibility is strong, and the method is suitable for mass production;

3. the impedance characteristic is excellent, and the transmission of an electrical stimulation signal is easy;

4. the skin is firmly contacted with the skin, and the motion artifact is not easy to generate;

5. compared with a gel wet electrode, the impedance characteristic is stable after long-term use.

Therefore, the electrode can make up the defect that the conductive gel wet electrode is used as an electrical stimulation electrode to a certain extent, and can exert the potential of the sensing wrist band body 203 to the maximum extent, and the scheme of the invention can achieve the following aims:

1. more accurate and flexible gesture control is realized;

2. more excellent sensory feedback, enhancing the immersion of the operator;

3. an electrode that more closely fits the skin, a wrist band with adjustability;

4. lower interface impedance, faster stimulus response;

5. stronger anti-interference performance;

6. lower cost.

When the virtual hand or the manipulator 201 is controlled by the gesture, if the virtual hand is the virtual hand, judging whether the virtual hand touches an object or not and the contact tightness degree of the virtual hand and the object are added into a software program; if the manipulator 201 is controlled by gestures, a mechanical sensor is mounted on the manipulator 201 to determine whether contact is made and how close the contact is made. The user wears the wrist strap, and embedded electrode of wrist strap and integration have signal processing and signal transceiver module. When the virtual hand or the manipulator 201 is in contact, the electrical stimulation parameters are coded according to the result of sensory signal analysis, then current pulses are sent to the stimulation electrodes, the electrodes release micro-current to stimulate the wrist of a user, so that the contact sensation of the palm and the fingers is induced and is transmitted to the brain of the user through the uplink nerve conduction path, the brain sends out a motion control instruction according to the received information and transmits the motion control instruction to the user through the downlink nerve conduction path to control the muscles of the virtual hand or the manipulator 201, and the sensory feedback is realized. In the human-computer interaction process, the accurate sensory feedback function when the virtual hand or different fingers of the manipulator 201 contact an object can be realized through corresponding different-point electrical stimulation.

The invention provides a human-computer interaction system and system facing gesture control, which realize real-time information feedback in gesture control through a neural feedback technology and achieve the purposes of closed-loop control and human-computer interaction. The feeling is an indispensable part in the precise motion control of human hands, and the human hands can still realize the gripping and the precise operation even under the condition of no vision. For a healthy human hand, in the process of moving and grabbing an object, mechanoreceptors on the skin of the hand respond to stimulation given by an interactive environment to acquire state information such as the position and posture of the hand and interactive information such as contact time, position and pressure with the object, afferent fibers transmit the sensory information to a brain through an ascending nerve conduction path, the brain integrates and analyzes the information, then generates a motion control command, and transmits the command to motor nerves of the hand through a descending nerve conduction path, so that the activities of muscles and joints of the hand are controlled and adjusted. In the invention, firstly, a pressure distribution signal of a contact object to a finger contact surface when a manipulator 201 provided with a flexible film pressure sensor 202 in a system grabs the object is acquired, then the current contact information category is identified through wireless transmission, signal processing and mode classification technology, an electrical stimulation parameter is coded by using the result of classification identification, a stimulation current parameter is set and is transmitted to a stimulation electrode in a wrist strap, and finally, the prepared sensing wrist strap body 203 based on the microneedle array type electrode is used for electrically stimulating the surface of skin to achieve the purpose of stimulating nerve endings, thereby realizing the nerve afferent of a sensory signal and providing a practical scheme for the human-computer interaction sensory feedback and the precise control of the manipulator 201.

The invention provides a gesture control-oriented man-machine interaction method, which comprises the following steps:

providing a gesture control-oriented human-computer interaction system; the man-machine interaction system facing gesture control comprises: the wrist strap comprises a wrist strap body 203, a micro-needle array electrode 204 arranged in the wrist strap body 203, a pressure sensor 202 arranged at each finger position of the manipulator 201, and a data processing and sending module 205 electrically connected with the micro-needle array electrode 204 arranged in the wrist strap body 203;

the pressure sensor 202 detects a contact signal between the manipulator 201 and an object, and transmits the detected contact signal to a data processing and transmitting module 205 arranged in the wrist strap body 203 in a wireless communication mode;

the data processing and transmitting module 205 analyzes the received signal and judges the type of the sensory information, and finally, the judgment result is used for encoding the electrical stimulation parameters, and the encoded current pulse is applied to the hand of the user through the micro-needle array electrode 204 on the wrist band body 203 to stimulate, induce and transmit the hand sensory information.

Although the invention has been described in detail above with reference to a general description and specific examples, it will be apparent to one skilled in the art that modifications or improvements may be made thereto based on the invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.