阅读说明：本技术 使交互式可穿戴设备能够作为听障个体的教育补充的系统和方法 (Enable interactive wearable device as the system and method for the education supplement for listening barrier individual ) 是由 奥农帕·巴蒂亚 克里希纳·马尔霍特拉 古里·莱克希米纳拉亚南 于 2018-03-03 设计创作，主要内容包括：本文描述了一种用于使交互式可穿戴设备能够作为听障个体的教育补充的系统和方法。该系统可以包括计算系统101、控制单元103和用户设备104。控制单元103可以嵌入在手套对上。控制单元103可以收集由用户执行的手势的原始数据,并通过用户设备104传输到计算系统101。计算系统101可以使用人工神经网络(ANN)方法处理原始数据。计算系统101可以导出手势字母的值并且还可以自动校正手势字母的值以便使误差最小化。计算系统101还可以在用户设备104的屏幕上显示图片或图形文字,以用于视觉提示个体使用手部手势字母拼写相关联的单词。(This document describes a kind of for enabling interactive wearable device as the system and method for the education supplement for listening barrier individual.The system may include computing system 101, control unit 103 and user equipment 104.Control unit 103 can be embedded in gloves to upper.Control unit 103 can collect the initial data of gesture performed by the user, and be transferred to computing system 101 by user equipment 104.Artificial neural network (ANN) method processing initial data can be used in computing system 101.Computing system 101 can export the value of gesture letter and can also automatically correct the value of gesture letter to make to minimize the error.Computing system 101 can also show picture or pictograph on the screen of user equipment 104, to use the associated word of hand gesture letter spelling for visual cues individual.)

1. a kind of system (100) for the education supplement for making interactive wearable device can be used as listening barrier individual, the system packet It includes:

Computing system (101)；

Control unit (103)；And

User equipment (104) is communicatively coupled with computing system (101) and control unit；

Wherein described control unit (104) further includes multiple sensors, microcontroller (201) and short range transceiver (204), wherein Described control unit (103) is communicatively coupled by short-range transmission protocols and user equipment (104), and wherein the multiple sensing Device is embedded in custom-made gloves to upper, wherein the custom-made gloves are to the form for being suitable for the hand using wearer, wherein the multiple biography Sensor includes one or more contact sensors (203) and one or more bending sensors (202), wherein described one A or more contact sensor (203) be configured to carry out the contact point of custom-made gloves pair the continuous scanning of predefined number with Just confirmation gesture is completed, and wherein the microcontroller (201) be configured to read the contact point of the custom-made gloves pair and The data of finger position from one or more bending sensor (202), wherein the microcontroller (201) configures For from a custom-made gloves of the custom-made gloves pair be directed to sign language gesture letter collect both hands initial data, so as to It completes to execute compared with the tentation data of gesture letter after the gesture of both hands, and wherein the microcontroller (201) is gone back It is configured so that short-range transmission protocols pass through user equipment (104) for original data transmissions to computing system (101)；And

Wherein the computing system (101) is configured to

The value of gesture letter is exported using neural network method processing initial data,

The data obtained from one or more bending sensors are compared with predefined data to confirm that gesture is alphabetical Value,

According to the value for automatically correcting gesture letter is compared to make to minimize the error；And

Show picture or pictograph, on the screen of user equipment (104) to use hand for visual cues individual/user The associated word of gesture letter spelling.

2. the system as claimed in claim 1, wherein the custom-made gloves weave to conductive sponge or steel reinforced concrete yarn is used, The conductive sponge or steel reinforced concrete yarn are programmed for for indicating the one or more of letter with sign language sign language at predefined position A contact sensor, wherein the sign language is India's sign language (ISL).

3. system as claimed in claim 2, wherein the conductive sponge or steel reinforced concrete yarn of predefined positioning be suitable for as base Other fiber compositions of fabric serve as one or more contact sensors together, other described fiber compositions are, for example, again Raw cellulose yarn, natural fiber, spandex and conductive yarn.

4. the system as claimed in claim 1, wherein each gloves of the custom-made gloves centering are configured so that short range communication Agreement communicates with one another.

5. system as claimed in claim 4, wherein the short-range communication protocol is bluetooth.

6. the system as claimed in claim 1, wherein the tentation data storage of the gesture letter is in the microcontroller.

7. the system as claimed in claim 1, wherein the initial data of the both hands includes from one or more sensing contacts The data of data and the finger position from one or more bending sensor that device obtains.

8. a kind of make interactive wearable device can be used as listening the method for educating supplement for hindering individual, wherein the method packet It includes:

By contact point of one or more contact sensors to custom-made gloves pair carry out the continuous scanning of predefined number with Just confirmation gesture is completed；

The contact point of the custom-made gloves pair and hand from one or more bending sensors are read by microcontroller Refer to the data of position；

It is collected from a custom-made gloves of the custom-made gloves pair for the gesture letter of sign language by the microcontroller double The initial data of hand, to be executed compared with the tentation data of gesture letter after the gesture for completing both hands；

Pass through user equipment for original data transmissions to computing system using short-range transmission protocols by the microcontroller；

Initial data is handled to export the value of gesture letter using neural network method by the computing system；

The data obtained from one or more bending sensors are compared with predefined data by the computing system The value of gesture letter for confirmation；

Make to minimize the error according to the value for automatically correcting gesture letter is compared by the computing system；

By the computing system, picture or pictograph are shown on the screen of user equipment, for visual cues individual Use the associated word of hand gesture letter spelling.

9. method according to claim 8, wherein the short-range communication protocol is bluetooth.

10. method according to claim 8, wherein the tentation data of the gesture letter stores in the microcontroller, and Wherein the sign language is India's sign language (ISL).

Technical field

Present invention relates in general to wearable device technical fields, and more particularly, to making interactive wearable device It can be as the system and method for the education supplement for listening barrier individual.

Background technique

According to " India's census: population enumeration data (ultimate population) in 2017 (Census of India: Population Enumeration Data (Final Population), 2017) report ", it has been observed that India various regions exist In 5-19 years old age group, about 1279358 significant hearing losses of children.Although wherein 67% going to school always, 21% never Obtain any regular education.In addition, nearly 3,000,000 hinder in total population for speech according to the census data (2011) of India Hinder and listen barrier.According to the resource of National Deaf Association (NAD), for disfluency and the sign language interpreters of barrier population is listened to only have 254 kinds of people.Children's hearing with serious/severe hearing loss (70db to -95db and 95dB or higher) is extremely difficult, and And depend on labiomaney and/or sign language.Sign language is three-level modelling apparatus by specific given deaf person group development, makes the group The member of body is stimulated from their prevailing circumstances.Sign language is a kind of visual space form, and space and movement can be with this Form is for indicating space and movement in the world in linguistics.Gap to the education for listening barrier children is for any country Obstruction that is sustainable and/or containing sexual development.Challenge is full of always to the education of hearing-impaired people.

Although using India's sign language (ISL) far from classroom many studies have shown that HI children are best by pasimology acquistion Predicament deprived the exchange power of a large amount of deaf populations.Since teacher is absorbed in the method for passing on orally, these deaf populations are in school for the deaf from 1- 12 grades almost without acquiring anything.When deaf children adheres to learning by sign language, their ignored and methods of passing on orally Continue, although they try by 12 grades, in them about 98% in illiteracy and they 2% almost without obtaining Knowledge is obtained, they are also faced with problem when looking for a job.(National Deaf Association, http://nadindia.org/).By expert Done about 10 classmans researches show that results: student can not even identify simple written words (such as comb or Sleep).Other than the school for the deaf for using bilingual method or comprehensive communication method to impart knowledge to students, the school for the deaf of India does not teach finger spelling. Finger spelling is a part of sign language.Research finds that deaf children can be by sign language culture English glossary and reading and writing Technical ability.When teacher imparts knowledge to students without using ISL and the method for passing on orally is not appropriately carried out or is not suitable for children, then the reading of children/ There will be no too many development for Writing Skill.Currently without the horizontal measuring system of the character learning specifically for deaf children.Although right Deaf children is also used in the various standardized character learning tests of hearing normal child exploitation.Although teachers know our ear The reading and writing technical ability of deaf student is very poor；But the empirical data for supporting this to put is considerably less.It is deaf due to not yet completing India at present The research of children's literacy rate, therefore the not statistical data about India's deaf children literacy rate.Everything be all based on experience and It observes and to be inferred to literacy rate very low.The low reason of deaf children literacy rate have it is several, including lack significant communication, Incomplete first language, quality of education difference etc..Deaf person will not be as Subjects With Normal Hearing with spoken contacts.But research also indicates that It is identical as the region that spoken language is embodied in brain to embody region in the brain for sign language for grammer and vocabulary.It is unique different Brain region it is related with limbs expression；It is spoken associated with sound and region of speaking, and the vision and sky of sign language and brain Between region it is associated.Therefore according to brain, sign language and it is spoken be equally language and it indicated in an identical manner.It is true On, sign language is the manual language of vision and deaf person is visual learning person.Compared with the people with Listening Ability of Ethnic, this also gives him Certain benefit relevant to vision and spatial analysis.For example, since deaf person pays attention to the variation in environment by observation, because The peripheral vision processing of this deaf person is more stronger than the people with Listening Ability of Ethnic.And deaf children has than the children with Listening Ability of Ethnic There is better visual space to remember.Because children absolutely not can exchange to any restrictions, pass through the study of ISL Facilitate all aspects of cognition and development of social psychology.

Also lack appropriate and effective audiovisual in the oral instruction education of these schools to support.This leads to most of deaf childrens Communication and language skill it is insufficient, cause the influence of the character learning technical ability difference of deaf person group.Reality is that school for the deaf does not use ISL mainly And nearly 5% deaf person attends school school for the deaf.The use of ISL is only limitted to vocational program and short course.Barrier is listened to learn for replenishing school The Current resource of raw teaching material learning stuff only limits/is limited to conventional teaching method.In addition it is also necessary to come from special teacher Or the help of resource teacher is to cultivate good reading and Writing Skill.

In current situations, by the help of teacher, these visual learning auxiliary tools help deaf children only to know Other object.But for that can spell out the words, understand them and then in order to which the academic requirement further read and write is established Knowledge obtained, they need local vector (India's sign language).For as reader, children must be learned by what they had known Mapping that is spoken and being printed upon between the word on the page.For English, as most of language, mapping is based on Sound.Severe deafness children are unfavorable as potential readers --- they can not easily obtain phonetic code and many people do not have Any language is better understood, let alone the language acquired in the printed article.Severe deafness children are unable to fully obtain For printed article-sound mappings sense of hearing basis.Therefore, the association between written word and spoken language must be taught, and described Association is obtained unlike as assuming in deaf person's education by children naturally.Technical ability of the children in structuring sign language provides language Speech basis, the development of another language skill can occur from the language basis, such as read.The experience of structuring sign language is formed Symbolic library, children can learn to print the meaning of word from the symbolic library.Showed in reading test preferably young Deaf children is that finger is had the ability or can write in terms of " association's technical ability " to spell the word to them and can translate Initialize the children of symbol.Finger spelling is the effective means to its writing communication technical ability of deaf children culture.Need to pass through religion Children are awarded structuring sign language form is established the language contacted with the positive literary style of English and reads the process of instruction to find and cultivate Associate technical ability.These technical ability are not that nature obtains, and must be formed by way of instruction.

In the prior art, the concept that whole-word used in every-day language is indicated using overall hand gesture, can use Symbol is absorbed in text or symbol to the conversion of speech in the teaching or study solution for listening barrier children, and vice versa.So And these solutions are only compatible with American Sign Language (ASL).In India, using India's sign language, wherein hand shape indicates reference object Visual space attribute, and formed based on hand shape and be different from the classifications of other classifications.This is it is desirable that classifier hand shape is language Element (in language can not divided the smallest syntactical unit, such as come, in etc.).The part ISL is by finger spelling system Britain's sign language and some other symbols influence, but it is most of unrelated with Europe notation.ISL itself is unique and one It causes.Most important theories in the presence of the unique features about ISL and by as the movement (Supalla 1986,1990 to originate from； Schick 1990；Liddell&Johnson 1987) and as origin hand shape (McDonald 1982；Engberg- Pedersen1993 the consistency in ISL) is described.The symbol of India's sign language (ISL) and other countries syntax, phonetics, Form and grammatically difference.ISL is using static and dynamic gesture, facial expression, head/body gesture, palmistry for body Position and orientation etc. indicate symbol.In ISL, sign language signifier's Selection utilization has/does not have the vocabulary of classifier hand shape Symbol constructs the sentence of the verb of expression movement and position.Classifier hand shape in the verb of movement and position is incorporated into verb In root, and the hand shape of verb is deleted since syllable constrains.ISL classifier on the whole, especially ISL reflect sign language table The interaction of the environment of the person of showing and their society, physics and function (as being proposed Denny1976).ISL using both hands with Represent some symbols, and even in finger spelling language place, personage or things nominal title.Only with American Sign Language (ASL) compatible existing solution may not support ISL, and the use of calculation translation ISL still be therefore unsolved Challenge.

Due to ISL mainly use both hands carry out sign language expression and finger spelling, a kind of system of long-term existence demand and Method, which becomes literate for the early stage of HI children is provided by the way that India's sign language is translated and/or decoded pair The interactive of India's sign language (ISL) learns by oneself and spells.

Summary of the invention

Before description this system and its application method, it should be understood that show because can have and be not known in the disclosure Multiple possible embodiments out, so the present disclosure is not limited to described specific device and its arrangements.It will also be appreciated that explanation Term used in book is only to describe particular version or specific embodiment, and be not intended to limit scope of the present application.The present invention Content is not intended to the essential feature for identifying this theme, is intended to be used to detect or limit the range of proposed theme.

In one embodiment, a kind of education benefit for making interactive wearable device can be used as listening barrier individual is disclosed The system filled.The system may include control unit, computing system and user equipment.User equipment can be with computing system and use Family equipment is communicatively coupled.Control unit may include multiple sensors, microcontroller and short range transceiver.In one embodiment In, control unit can be communicatively coupled by short-range transmission protocols and user equipment.In one embodiment, multiple sensors can To be embedded in custom-made gloves to upper.The custom-made gloves are to the form that may be adapted to the hand using wearer.Multiple sensors can wrap Include one or more contact sensors and one or more bending sensors.One or more contact sensors can be with The continuous scanning gesture for confirmation that pre- justice determines number is carried out to the contact point of the custom-made gloves pair to complete.Microcontroller can be read Take the contact point of the custom-made gloves pair.Microcontroller can read the finger position from one or more bending sensors Data.Microcontroller can collect both hands for the gesture letter for sign language from a custom-made gloves of the custom-made gloves pair Initial data, so as to complete both hands gesture after execute compared with the tentation data of gesture letter.Microcontroller is also Short-range transmission protocols can be used and pass through user equipment for original data transmissions to computing system.Nerve can be used in computing system Network method handles initial data to export the value of gesture letter.Computing system can also will be passed from one or more bendings Sensor data obtained are compared the value to confirm that gesture letter with predefined data.Computing system can also basis This automatically corrects the value of gesture letter relatively to make to minimize the error.Computing system can also be shown on the screen of user equipment Diagram piece or pictograph, to use the associated word of hand gesture letter spelling for visual cues individual.

In another embodiment, a kind of education for making interactive wearable device can be used as listening barrier individual is disclosed The method of supplement.This method may include being made a reservation for by contact point of one or more contact sensors to custom-made gloves pair The continuous scanning of adopted number gesture for confirmation is completed.This method, which may also include, reads the custom-made gloves pair by microcontroller The data of contact point and the finger position from one or more bending sensors.This method, which may also include, passes through microcontroller Device, from a custom-made gloves of the custom-made gloves pair for sign language gesture letter collect both hands initial data, so as to It completes to execute compared with the tentation data of gesture letter after the gesture of both hands.This method can also include passing through microcontroller Device passes through user equipment for original data transmissions to computing system using short-range transmission protocols.This method may include passing through meter Calculation system handles initial data using neural network method to export the value of gesture letter.This method may include passing through meter The data obtained from one or more bending sensors are compared with predefined data to confirm that gesture word by calculation system Female value.This method may include keeping error minimum according to the value that automatically corrects gesture letter is compared by computing system Change.This method may include showing picture or pictograph on the screen by computing system, to make for visual cues individual With the associated word of hand gesture letter spelling.

Detailed description of the invention

It is described hereinbelow in detail with reference to the accompanying drawings.In the accompanying drawings, it is first to identify the appended drawing reference for the leftmost number of appended drawing reference The attached drawing of secondary appearance.The same numbers used through attached drawing indicate identical feature and component.

Fig. 1 show it is according to the embodiment of this theme, for making interactive wearable device can be used as listening barrier individual Education supplement system 100 web implementations.

Fig. 2 (A) show it is according to this theme, for the palm and the back of the hand of the right hand and left hand, for indicating every to sign language The position of one or more contact sensors and one or more bending sensors that a letter (A-Z) is marked.

Fig. 2 (B) shows the component of control units 103 according to this theme, in the wearable gloves of insertion.

Fig. 2 (C) show it is according to the exemplary embodiment of this theme, use tubular braided structure and as single knitting One or more contact sensor points of cotton/steel conductive yarn are integrated in gloves panel by the base fabric of structure.

Fig. 3 shows the component of the computing system 101 according to the embodiment of this theme.

Fig. 4 (A) shows the frame of the artificial neural network (ANN) for character recognition of the embodiment according to this theme Figure.

Fig. 4 (B) shows artificial neural network (ANN) classifier figure of the embodiment according to this theme.

Fig. 5 show it is according to the embodiment of this theme, in the training stage using artificial neural network processing input number Just the step-by-step movement flow chart that implementation pattern is identified and automatically corrected accordingly.

Fig. 6 show it is according to the embodiment of this theme, handled in neural network in Qualify Phase input data with Just the step-by-step movement flow chart automatically corrected is realized.

Fig. 7 shows the method 700 of the embodiment according to this theme implemented by system 100.

Fig. 8 shows the exemplary embodiment of the system 100 according to the embodiment of this theme.

Fig. 9 shows the learning functionality of the first exemplary embodiment according to this theme executed by system 100.

Figure 10 and Figure 11 shows spelling and word according to second exemplary embodiment of this theme by computing system 100 It converges and constructs.

Figure 12 and Figure 13 shows the automatic school by 100 pairs of words of the third exemplary embodiment according to this theme Just.

Figure 14 shows the picture visual on a display screen or pictograph of the embodiment according to this theme, the picture Or pictograph prompt individual uses the associated word of hand gesture letter spelling.

Specific embodiment

It will be discussed in detail the embodiment of its all feature of some explanations of the disclosure now.Word "comprising", " having ", " containing " and " comprising " and its other forms are intended to equivalent in the sense and are Open ends, because of these lists One or more after any one of word are not meant to be one or more full list, or do not mean that only It is one or more listed by being limited to.

It must further be noted that unless the context clearly determines otherwise, otherwise singular " one ", "one" and " described " Including plural.Although similar or equivalent any method can be used in embodiment of the disclosure with approach described herein Practice or test, but illustrative methods are described now.The disclosed embodiments are only the exemplary implementation of the disclosure Example can embody in a variety of manners.

The various modifications of embodiment will be apparent to those skilled in the art, and general herein Principle can be applied to other embodiments.However, one of those skilled in the art will readily recognize that, the disclosure is not intended to be limited to The embodiment shown in, but meet the broadest range being consistent with the principles described herein and feature.

This theme is related to system and the side of the education supplement for making interactive wearable device can be used as listening barrier children Method.With reference to Fig. 1, according to the embodiment of this theme, show for making interactive wearable device can be used as listening barrier individual Educate the web implementations of the system 100 of supplement.As shown, control unit 103 and user equipment 104 are communicatively coupled.Control Unit 103 processed can be embedded in wearable gloves (Fig. 2 C).In one embodiment, control unit 103 can be for example, by indigo plant The communication protocols such as tooth, USB are communicated with user equipment 104.In addition, user equipment 104 can be communicatively coupled with computing system 101.

Although it is contemplated that implement computing system 101 on the server to explain this theme, but it is understood that, it calculates System 101 can also be implemented in various computing systems, such as laptop computer, desktop computer, notebook, work station, large size Computer, server, network server etc..It will be understood that can be set by multiple users by one or more users Standby 104-1,104-2...104-N (referred to generically hereinafter as user 104) or the application being present on user equipment 104 come to visit and receive customers Calculation system 101.The example of user equipment 104 can include but is not limited to portable computer, personal digital assistant, handheld device And work station.User equipment 104 is communicatively coupled to computing system 101 by network 102.

In one embodiment, network 102 can be wireless network, cable network or combinations thereof.104 energy of user equipment It is enough to access network 102 using the wired or wireless network attachment device for including the communication technology updated.Network 102 can be implemented For one of different types of network, such as Intranet, local area network (LAN), wide area network (WAN), internet etc..Network 102 It can be dedicated network or shared network.Shared network representation uses such as hypertext transfer protocol (HTTP), biography transport control protocol The association for the different type network that the various agreements such as view/Internet protocol (TCP/IP), Wireless Application Protocol (WAP) communicate with one another. In addition, network 102 may include the various network equipments, including router, bridge, server, calculating equipment, storage equipment Deng.

It is shown one or more in the wearable gloves of insertion referring now to Fig. 2 (A) according to the embodiment of this theme The position of contact sensor and one or more bending sensors.In one embodiment, one or more contacts pass Sensor and one or more bending sensors can have the position in the mark point of the palm and the back of the hand in left hand and the right hand It sets to indicate each alphabetical (A-Z in ISL) for sign language.In one embodiment, mark point can indicate to mention to being embedded in For the custom-made gloves that are directly contacted with wearer skin to upper one or more contact sensors and one or more The predefined positioning of bending sensor.In one embodiment, the custom-made gloves pair are woven using conductive sponge or steel reinforced concrete yarn. In one embodiment, use regenerated cellulose yarn as base fabric, the conductive sponge or steel reinforced concrete yarn of predefined positioning It is adapted to act as one or more contact sensors.In one embodiment, can by fusion fabric will be one or more Bending sensor is embedded in and is sewn on gloves.

Referring now to Fig. 2 (B), the control unit 103 being embedded in wearable gloves is illustrated according to the implementation of this theme Component.As shown in Fig. 2 (B), control unit 103 includes multiple sensors, microcontroller 201, tactile actuator 205, short distance Transceiver 204, power supply unit 206 and USB port 207.In one embodiment, multiple sensor may include one or More bending sensors 202 and one or more contact sensors 203.In one embodiment, this is one or more A bending sensor 202 can be resistance type sensor.The resistance of one or more bending sensors 202 can follow finger It is each bending and change.In one exemplary embodiment, which can have comfortably Ground is maintained on the finger of gloves and the simple structure with long life cycle.In one exemplary embodiment, this Or more bending sensor 202 may include 10K ohm of flat resistance and 60K to 110K ohm of bending resistance.This Outside, which can obtain very big form factor.In one embodiment, this or The bending that the second joint and articulations digitorum manus of each finger are measured on each finger can be set in more bending sensors. In one embodiment, when making digital flexion for all alphabetical (A-Z in ISL) every time with various angle of displacements, this Or more bending sensor 202 may be adapted to measure and provide close result.

In one embodiment, cotton/steel conductive yarn mixture can be used in one or more contact sensors 203 It is made.In one embodiment, yarn can be flexible, it is durable and at predefined position with other fiber compositions It is woven together to serve as contact sensor.In one embodiment, when the contact of two different pieces of gloves, one or more Each of multiple contact sensors 203 can show as simple switch and provide input to microcontroller 201.At one In exemplary embodiment, conductive yarn can be programmed for contact sensor at predefined position, wherein tubular braided structure and Base as regenerated cellulose yarn is used together.

In one exemplary embodiment, CONDUCTIVE TEXTILE YARN line is used for so that sensor is compatibly integrated with gloves.One In a little embodiments, fabric can be manufactured in gloves by the electrical characteristics and advanced knitting skill using soft textile material and sensed Device.In one embodiment, by using knitting skill, adherency in the braiding CONDUCTIVE TEXTILE YARN line at the appropriate area of gloves Weaving electrode can be in close contact with wearer skin.In one embodiment, the mixture of textile fabric yarn can be Woven at the specific region of hand, with remember children or user under India's weather conditions to the use of gloves, the sweat gland in hand, and And obtain the accuracy of improved performance and result.In one exemplary embodiment, textile fabric yarn can be by natural fine Dimension, two kinds of regenerated celulose fibres, spandex and conductive yarn are made.In one embodiment, textile fabric yarn can provide Being comfortable on property and hot physiological comfort, this can further help in stable heat transfer and moisture transmitting.It can basis Project/user requires to weave textile fabric yarn on the specific area.In one embodiment, the braiding of various scribbleds connects With surface layer gloves can be applied at the reserved area that the maximum in hand region perspires together.In one embodiment, hand Set can contribute to being comfortable on property and hot physiological comfort, this further helps in stable heat transfer and moisture transmitting.

Referring now to Fig. 2 (C), according to the exemplary embodiment of this theme, shows and use tubular braided structure and as list One or more contact sensor points of cotton/steel conductive yarn are integrated in gloves panel by the base fabric of face knitted structure In.Referring now to Fig. 2 (c) (i), shows and one or more contact sensor points of cotton/steel conductive yarn are integrated in the right hand In facies palmaris plate.In one exemplary embodiment, regenerated cellulose yarn 209 can be base fabric, and conductive yarn 208 It can be programmed to act as contact point.Referring now to Fig. 2 (c) (ii), show one or more contacts of cotton/steel conductive yarn Sensor points are integrated in right hand backplate.In one exemplary embodiment, natural fiber, regenerated cellulose yarn, spandex 211 fiber composition can be base fabric, and conductive yarn 210 can be programmed to serve as contact point.Referring now to Fig. 2 (c) (iii) shows and one or more contact sensor points of cotton/steel conductive yarn is integrated in left hand facies palmaris plate.In In one exemplary embodiment, regenerated cellulose yarn 213 can be base fabric, and conductive yarn 212 can be programmed for Serve as contact point.Referring now to Fig. 2 (c) (iv), show one or more contact sensor point sets of cotton/steel conductive yarn At in back of right hand plate.In one exemplary embodiment, the fiber group of natural fiber, regenerated cellulose yarn, spandex 215 Closing object can be base fabric, and conductive yarn 214 can be programmed to act as contact point.

In one embodiment, one or more contact sensors 203 are configurable to the contact to custom-made gloves pair The continuous scanning gesture for confirmation that point carries out predefined number is completed.In one embodiment, one or more contacts pass Sensor 203 can execute the scanning to each contact point.In one exemplary embodiment, one or more contact sensors 203 may need 20 milliseconds to scan each contact point.In one embodiment, can compare related with continuous scanning three times Data are to confirm that gesture is completed.In one embodiment, the data between continuous scanning relatively can be used for individually acquisition pair In the constant data of every hand.In one exemplary embodiment, it when hand is moved because doing gesture, is obtained by contact point Data can continue to change.Therefore, necessary by contact point data obtained for continuous scanning three times when gesture is completed Keep identical/constant.Therefore, one or more contact sensors 203 can consume 60 milliseconds so that obtain from contact point can be with The final data completed for confirming gesture.It in one exemplary embodiment, is 6 from the size of contact point data obtained Byte, i.e., every 48, hand.

In one embodiment, microcontroller 201 is configurable to read contact point and the finger position of custom-made gloves pair Data.In one embodiment, one or more bending sensors 202 can obtain the data of finger position, such as hand The bending angle of finger.In one exemplary embodiment, microcontroller 201 may need 10 milliseconds to read contact point and bending Sensor.

In one embodiment, microcontroller 201 is configurable to be directed to from a custom-made gloves of custom-made gloves pair The gesture letter of sign language collects the initial data of both hands.In one embodiment, sign language can be India's sign language (ISL).One In a embodiment, initial data be can be from one or more contact sensors and one or more bending sensors institute The data of acquisition.In one embodiment, after the gesture for completing both hands, microcontroller can execute initial data and gesture The comparison of the tentation data of letter.In one exemplary embodiment, left hand custom-made gloves can be by original data transmissions to the right side Hand custom-made gloves.In one embodiment, after gesture completion, initial data can be with compared with alphabetical tentation data Right hand data carry out.In one exemplary embodiment, microcontroller 201 can be 8 PIC18F87K22 microcontrollers, Program storage with 16K or more, 4K RAM.In one embodiment, microcontroller may include 2 serial ports (that is, RS232 compatible), 5 simulation inputs, 10 ADC module and 50 or more number i/o pin.Implement at one Example in, microcontroller 206 can with 207 electronics coupled of short range transceiver 204 and USB port, so as to respectively use short range communication Agreement or USB port 207 promote original data transmissions to user equipment 104.In one exemplary embodiment, short distance is received and dispatched Device 204 can be bluetooth module, and short-range communication protocol can be Bluetooth protocol.In one embodiment, short range transceiver 204 can be used for the communication between both hands and the communication with computing system 101.In one embodiment, computing system 101 can To implement on user equipment 104.As shown in Figure 2, control unit 103 can also include the electricity powered to microcontroller 201 Source unit 206.In one exemplary embodiment, microcontroller 201 can be used the control of two-way programming input and output (PIO) with Server-host mode is served as between two pairing bluetooths.In one embodiment, microcontroller 201 can be received successfully Collect and handle the data of one or more contact sensors between bimanual input or one or more bending sensors, and And it is communicated using inquiry response method (server-host mode) with system 101.

Referring now to Fig. 3, the component of computing system 101 is illustrated according to the implementation of this theme.As shown in Figure 3, it calculates System 101 may include at least one processor 301, input/output (I/O) interface 302, memory 303,304 sum number of module According to 309.In one embodiment, which is configurable to extract and execute to be stored in memory 303 Computer-readable instruction.

In one embodiment, I/O interface 302 may include various software and hardware interfaces, for example, network interface, figure Shape user interface etc..I/O interface 302 can permit computing system 101 and interact with user equipment 104.In addition, I/O interface 302 can So that user equipment 104 can be with other calculating equipment of such as network server and Outside data services device (not shown) etc Communication.I/O interface 302 can promote a variety of communications in various networks and protocol type, including such as LAN, electricity The wireless network of the cable network of cable etc. and such as WLAN, honeycomb or satellite etc.I/O interface 302 may include for connecting To one or more ports of another server.

In one embodiment, memory 303 may include any computer-readable medium as known in the art, example Volatile storage as included such as static random access memory (SRAM) and dynamic random access memory (DRAM) etc Device and/or such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disk, CD and storage card etc it is non-volatile Property memory.Memory 303 may include module 304 and data 309.

Module 304 includes executing particular task, function or routine, programs, objects, the group of implementing particular abstract data type Part, data structure etc..In one embodiment, module 304 may include pattern recognition module 305, automatically correct module 306, generation module 307 and other modules 308 are fed back.Other modules 308 may include supplementing application and the function of user equipment The program or coded command of energy.

In one embodiment, data 309 may include repository 310 and other data 311.In an exemplary implementation In example, repository 310 is configurable to one or more handled, receptions stored by module 304 and the data generated. Other data 311 may include due to the execution of one or more modules data generated.

In one embodiment, user equipment 104 can be used by I/O interface 302 to 101 note of computing system in user Volume.User can be used be mounted on user equipment 104 application so as to computing system 101 register.

In one embodiment, pattern recognition module 305 can be to from carrying out from the received initial data of control unit 103 Reason.In one embodiment, artificial neural network (ANN) method processing initial data can be used in pattern recognition module 305.In In one embodiment, artificial neural network (ANN) can be used for exporting the value of gesture letter.In one embodiment, mode is known Other module 305 is configurable to for the data obtained from one or more bending sensors being compared with predefined data. In one embodiment, predefined data can be stored in the data in the database of computing system 101.In one embodiment In, pattern recognition module 305 can confirm the value of gesture letter according to comparing.

Referring now to Fig. 4 A, the artificial neural network (ANN) for pattern-recognition is illustrated according to the implementation of this theme Block diagram.In one embodiment, microcontroller 201 can will from one or more bending sensors 202 and one or more The original data transmissions of multiple contact gestures obtained of point sensor 203 are to ANN classification device.Referring now to Fig. 4 B, according to this master The implementation of topic is illustrated ANN classification device figure.In one embodiment, ANN can include ten input nerves in input layer Member.In one exemplary embodiment, ANN may include from left thumb (LT), left index finger (LI), left finger (LM), Zuo Wu Name refers to (LR), left little finger of toe (LL), right thumb (RT), right index finger (RI), right middle finger (RM), the right third finger (RR), right little finger of toe (RL) Input.In one embodiment, ANN can include four neurons in hidden layer.In one embodiment, ANN can be with It include 26 output neurons in output layer.In the exemplary embodiment of this theme, ANN parameter institute in table 1 below Show.

Table 1

Referring now to Fig. 5, be illustrated in the training stage according to the implementation of this theme handled using artificial neural network it is defeated Enter the step-by-step movement flow chart that data are identified and automatically corrected so as to implementation pattern.At step 501, pattern recognition module 305 can The initial data obtained from one or more bending sensors is marked and is stored to be configured to execute instruction In X position.At step 502, pattern recognition module 305 is configurable to execute instruction to separate target and input.In In one embodiment, desired output of the object representations for given input.At step 503, pattern recognition module 305 can match It is set to and executes instruction so that target and input are divided into ten groups and are used for ten folding cross validations.At step 504, pattern-recognition mould Block 305, which is configurable to execute instruction, initializes neural network will pass through parameter shown in selection table 1.In step 505 Place, pattern recognition module 305 are configurable to train neural network using first group of labeled data in ten groups.In step At rapid 506, pattern recognition module 305 is configurable to test neural network using the labeled data for nine groups of residue. At step 507, pattern recognition module 305 is configurable to creation and reads letter character/reading letter function, with from step Letter character/letter is read in classification specified by neural network in 506.At step 508, pattern recognition module 305 can To be configured to storage neural network.At step 509, pattern recognition module 305 is configurable to identify to for character/letter Score/accuracy calculated.At step 510, pattern recognition module 305 is configurable to be iteratively repeated step 505- 509 to complete ten folding cross validations.At step 511, pattern recognition module 305 is configurable to the mind with largest score Through network creation function my_nn.

Referring now to Fig. 6, according to the embodiment of this theme, shows and handle input number in neural network in test phase Just the step-by-step movement flow chart that implementation pattern is identified and automatically corrected accordingly.At step 601, pattern recognition module 305 can be with It is configured to obtain unlabelled bending sensor data and be stored in (Xtest).At step 602, pattern recognition module 305 are configurable to be invoked at the function my_nn created in step 511 (as shown in Figure 5) in the training stage and incite somebody to action Xtest is transmitted as function parameter.At step 603, pattern recognition module 305, which is configurable to call, reads letter character (read Letter) or alphabetical (read alphabet) function is read, to identify from the classification as specified by function my_nn Character.

Referring now again to Fig. 3, automatically corrects module 306 and be configurable to automatically correct gesture letter according to comparing Value.In one embodiment, automatically correcting module 306 can be used minimizing the error and automatically correcting word for family.One In a embodiment, feedback generation module 307 is configurable to generate the feedback for hand gesture letter or word to user.In In one embodiment, automatically corrects module 306 and be configurable to show picture or pictograph on the screen.In one embodiment In, screen-vision prompts individual/user to use the associated word of hand gesture letter spelling.

Referring now to Fig. 7, according to the embodiment of this theme, the method 700 implemented by system 100 is shown.In a reality It applies in example, user can be with wearing gloves pair.In one embodiment, it is internal can be embedded in the gloves for osculating element 103.

At step 701, one or more contact sensors 203 in control unit 103 can be to the custom-made gloves Pair contact point execute the continuous scanning of predefined number.

At step 702, microcontroller 201 is configurable to read the contact point of the custom-made gloves pair and finger position Data.In one embodiment, one or more bending sensors 202 can obtain the initial data of finger position.

At step 703, microcontroller 201 can be directed to the hand of sign language from a custom-made gloves of the custom-made gloves pair The initial data of one or more bending sensors of gesture letter collection both hands and one or more contact point sensors Data.In one embodiment, sign language is India's sign language (ISL).In one embodiment, two gloves pass through short range communication Agreement is coupled with each other.In one exemplary embodiment, two-way programming input and output can be used in microcontroller 201 (PIO) control is to serve as server-host mode between two pairing bluetooths.In one embodiment, initial data can be with The ASCII fromat for communication is converted to, and is converted to again in host to calculate as purpose hexadecimal format.Micro-control Initial data can be compared by device processed after the gesture for completing both hands with the tentation data of gesture letter.

At step 704, short-range transmission protocols are can be used by original data transmissions to user equipment in microcontroller 201 104。

At step 705, neural network method processing initial data is can be used to export gesture word in user equipment 104 Female value.In one embodiment, user equipment 104 can be handled by pattern recognition module 305 using neural network method Initial data.

At step 706, user equipment 104 can by pattern recognition module 305 by the data obtained from contact point with Predefined data is compared the value to confirm that gesture letter.

At step 707, user equipment 104 can automatically correct gesture word according to comparing by automatically correcting module 306 Female value is so as to minimize the error.

At step 708, user equipment 104 can show picture or pictograph on the screen.In one embodiment, User equipment 104 can show picture or pictograph by feedback generation module 307.Picture or the figure text shown on screen Word visual cues individual uses the associated word of hand gesture letter spelling.

Referring now to Fig. 8, the exemplary embodiment of system 100 is shown according to this theme.In one embodiment, Yong Huke To use user equipment 104 to access computing system 101.User can be used user equipment 104 and pass through I/O interface 302 to calculating System 101 is registered.User can be used be mounted on user equipment 104 application so as to computing system 101 register.At one In embodiment, user can be children.At step 801, children can star the application being mounted on user equipment 104.In Step 802 place, computing system 101 can prompt children's wearing gloves by user equipment 104.At step 803, system 101 It can check children's whether wearing gloves.If children do not have wearing gloves, computing system 101 repeats step 802.If Children's wearing gloves, then at step 804, computing system 101 can prompt children to keep gesture 5 seconds.At step 805, System 101 can detecte gesture performed by children, if children are not carried out gesture, at step 806, and the control of gloves Unit 103 can provide touch feedback to children, and method 800 returns to step 804 place.If children execute gesture, Start countdown 5 seconds in the display/screen of user equipment 104.At step 807, computing system checks that children whether will Gesture is kept for 5 seconds.If children do not keep gesture 5 seconds, at step 808, the control unit 103 of gloves can be to youngster Child provides touch feedback, and method 800 returns to step 804 place.If children keep gesture 5 seconds, in step 809 Place, system 101 can show confirmation/congratulations message to children.At step 810, system 101 can prompt children from study function One function of selection in function is spelt in energy, inquiry/zero offset capability.If children select learning functionality, computing system into Row arrives step 811.If children select inquiry/zero offset capability, computing system proceeds to step 812.If children select Spelling function is selected, then computing system proceeds to step 813.

Referring now to Fig. 9, the study function as performed by system 100 is shown according to first exemplary embodiment of this theme Energy.In one embodiment, children can be with wearing gloves pair, and wherein control unit 103 is embedded in the gloves to upper.In step 901 Place, computing system 101 can show letter/Alphabet Gesture video to children.In one exemplary embodiment, computing system 101 can show letter/Alphabet Gesture video on user equipment 104.At step 902, children can imitate gesture.In step At rapid 903, the control unit 103 for being embedded in the gloves centering can sense the gesture executed by children.In one embodiment, it controls The data sensed can be transferred to computing system 101 by unit 103 processed.At step 904, computing system 101 can be identified Mode in gesture.At step 905, whether computing system 101 can correct with checking mode.If mode is incorrect, Step 906 place, the control unit 103 of gloves give children's touch feedback to correct gesture.At step 907, children self It corrects the gesture and computing system returns to step 905 place to verify gesture.If gesture mode is correct, in step 908 Place, computing system 101 can show visual feedback to children, and computing system returns to step 901 place to learn new word It is female.

0 and Figure 11 referring now to figure 1 shows and is promoted by computing system 100 according to second exemplary embodiment of this theme Spelling and vocabulary building.Referring now to figure 10, at step 1001, children can choose spelling function.At step 1002, it is System 101 can show that manpower language indicates word/dash line picture/video to children.In one exemplary embodiment, it calculates System 101 can prompt picture on the screen.In a further exemplary embodiment, computing system 101 can be shown just in sign language Indicate the video of the people of word.In yet another exemplary embodiment, computing system 101 can be on the screen of user equipment 104 Show dash line.At step 1003, computing system 101 can prompt children to carry out hand to the word of the picture shown on screen Refer to spelling.At step 1004, system 101 can check whether children keep gesture 3 seconds.If it is not, then computing system 101 return to step 1003 place.If children keep gesture 3 seconds, at step 1005, system 101 can check gesture Whether can recognize.If gesture not can recognize, at step 1006, the control unit 103 of gloves can provide touch feedback, And computing system 101 shows the vector watermark of question mark on the screen of user equipment 104.At step 1007, computing system 101 can make vector watermark disappear.At step 1024, system 101 can check gesture whether continuous 3 times it is unidentified.If No, then computing system 101 may return to step 1003 place.If gesture continuous 3 times are unidentified, at step 1025, meter Calculation system 101 can show study course to user.In one exemplary embodiment, study course may include for showing that finger is spelt Image frame.If gesture be it is identifiable, at step 1008, computing system 101 can show letter vector water Print and letter.At step 1009, computing system 101 can show letter above dash line, and then make vector water Print disappears.At step 1010, computing system 101 can execute the verification process of letter.If alphabetical invalid, in step At 1011, computing system 101 is it can be concluded that the incorrect conclusion of letter that finger is spelt.At step 1012, the control of gloves Unit 103 can provide touch feedback, and computing system 101 shows visual feedback.At step 1013, computing system 101 " thumb is downward " symbol can be shown below dash line.At step 1014, computing system 101 can check finger spelling Whether letter is continuous incorrect three times.If it is not, computing system returns to step 1003 place.If it is, in step 1015 Place, computing system 101 can show the option of study course and prompt to user.If user selects prompt options, in step At 1016, computing system 101 can be to user's display reminding.In one exemplary embodiment, prompt can indicate to include just The really frame of letter.Alternatively, if user selects study course option, at step 1017, computing system 101 can be to user Show study course.In one exemplary embodiment, study course can indicate include finger spelling image frame.If computing system 101 execute the verifying of letter and show that the letter of finger spelling is correct conclusion, then at step 1018, computing system 101 " thumb is upward " symbol can be shown below dash line.At step 1019, computing system 101 can check new in word Letter.If there are next letter in word, computing system 101 may return to step 1003.If there is no Next letter, then at step 1020, computing system 101 is it can be concluded that this is the conclusion of the ending of word.In step 1021 Place, computing system 101 can show confirmation or congratulations message to user.At step 1022, computing system 101 can be checked down One word.If so, computing system 101 returns to step 1002 place.If it is not, then terminating process at step 1023.

Referring now to figure 11, the spelling function as performed by system 100 is shown according to second exemplary embodiment of this theme Energy.In one embodiment, children can be with wearing gloves pair, and wherein control unit 103 is embedded in the gloves to upper.In step 1101 Place, computing system 101 can show picture/word gesture video to children.In one exemplary embodiment, computing system 101 can show picture/word gesture video on user equipment 104.At step 1102, children can be to each of word Letter executes gesture.At step 1103, the control unit 103 of insertion gloves centering can sense the gesture executed by children. In one embodiment, the data sensed can be transferred to system 101 by control unit 103.At step 1104, system is calculated System 101 can identify the mode in gesture.At step 1105, whether computing system 101 can correct with checking mode.If mould Formula is incorrect, then at step 1106, the control unit 103 of gloves gives children's touch feedback to correct gesture.In step At 1107, children's self-correction gesture and computing system 101 return to step 905.If gesture mode is correct, in step At rapid 1108, computing system 101 can show visual feedback to children, and the picture of prompt is spelt back to step 1101 Or the word of gesture video.

2 and Figure 13 referring now to figure 1 shows the list realized by system 100 according to the third exemplary embodiment of this theme Word automatically corrects.Referring now to figure 12, at step 1201, children can choose inquiry/zero offset capability.In step 1202 Place, computing system 101 can prompt children to carry out finger spelling.In one embodiment, children can carry out word Finger spelling.At step 1203, computing system 101 can check whether children keep gesture 3 seconds.If it is not, then meter Calculation system 101 may return to step 1202 place.If children keep gesture 3 seconds, at step 1204, computing system 101 can check whether gesture can recognize.If gesture not can recognize, at step 1205, the control unit 103 of gloves can To provide touch feedback, and computing system 101 shows the vector watermark of question mark on the screen of user equipment 104.In step At 1206, computing system 101 can make vector watermark disappear.At step 1207, system 101 can check whether gesture is continuous 3 times unidentified.If it is not, then computing system 101 may return to step 1202.If gesture continuous 3 times unidentified, in step At rapid 1208, computing system 101 can show study course to user.In one exemplary embodiment, study course can indicate to be used for Show the frame of the image of finger spelling.If gesture be it is identifiable, at step 1209, computing system 101 can be shown The vector watermark of letter and letter.At step 1210, computing system 101 can show letter above dash line, and Vector watermark is then set to disappear.At step 1211, computing system 101 can check similar to word in the database is stored 2+ letter sequence.At step 1212, computing system 101 can show word on the screen.At step 1213, meter Calculation system 101 can make children it can be seen that and selecting word.At step 1214, computing system 101 can check next A word.If next word exists, computing system 101 returns to step 1202.If it does not exist, then in step 1215 Locate end process.

Referring now to figure 13, according to the third exemplary embodiment of this theme show the inquiry as performed by system 100/from Dynamic calibration function.In one embodiment, children can be with wearing gloves pair, and wherein control unit 103 is embedded in the gloves to upper.In Step 1301 place, children can execute gesture to each of word letter.At step 1302, it is embedded in the control list of gloves centering Member 103 can sense the gesture executed by children.In one embodiment, control unit 103 can pass the data sensed It is defeated to arrive computing system 101.At step 1303, computing system 101 can identify the mode in gesture.At step 1304, meter Whether calculation system 101 can correct with checking mode.If mode is incorrect, at step 1305, computing system 101 can be mentioned For the close suggestion of the word indicated with sign language.At step 1306, children's self-correction gesture and computing system return To step 905.If gesture mode is correct, at step 1307, computing system 101 can show visual feedback to children, And back to step 1301 to execute the gesture for new word.

Referring now to figure 14, show the display screen for spelling associated word using hand gesture letter in prompt individual Upper visual picture or pictograph.As shown in Figure 14, picture or pictograph include the word indicated by user's sign language In, the hand gesture of India's sign language (ISL) letter and from Figure 14 it can be seen that the associated figure for the word Piece (such as potato, beet root and carrot etc.).

Exemplary embodiment can provide certain advantage as discussed above.Some embodiments of the present disclosure may be implemented System and method more accurate than conventional system.The precision of system is realized based on two layers of sensor is used.By using nerve The combination of network analyzing data, sensor can make system more accurate.

Detection may be implemented as the friendship between the different piece of the hand of sign language signifier in some embodiments of the present disclosure The system and method for mutual gesture.

Some embodiments of the present disclosure may be implemented for aims of education system and method, not just for pass student For disabled students, barrier student/individual is especially listened.

Some embodiments of the present disclosure may be implemented to be specifically designed to remember the usability of user and the intensity of safety Product.System executes heavy processing in a user device, further realizes reduction hardware cost.

In some embodiments, the wearable gloves disclosed in the disclosure can have to wearable device sensor system System realizes that safely and reliably output is and its necessary special designs and non-invasive sensors are integrated.The design, which meets, to be used ISL is converted to the precision of text in the children gloves for the education for listening barrier children.

In some embodiments, system includes wearable device sensing system, the wearable device sensing system packet Include various component sensors, wearable material, intelligent textile, actuator, power supply, wireless communication module and link, Control and processing unit, user interface, software and the advanced algorithm for data extraction and decision.Appearance should not only meet beauty It learns and requires, and the weight of system and size should be made small and it should not interfere the activity and movement of user, this makes in children It is essential in the case where the gloves.

System and method may be implemented to provide simple, attractive and effective interface in some embodiments of the present disclosure The thing to be completed is conveyed to children/individual.That system can be provided in is whenever can referring to, how sign language indicates every The study course of a letter character.In one embodiment, can be provided by interface has identical visual language and can be by youngster The visual cues that virgin/individual is accurately explained.

If some embodiments of the present disclosure can provide sign language expression letter character correctly or incorrectly guide children/ The feedback mechanism of individual.

In some embodiments, it can indicate/draw that a certain group of letter character is to calibrate by user hand language to be looked for novelty System.This process can help to understand the full breadth of movement and new user's sign language expression/or the ad hoc fashion for drawing character.

It can be independently or in any combination way using including aforementioned paragraphs, the embodiment of description, example and substitution Any embodiment, example and alternative solution in the various aspects of scheme or corresponding individually feature.It is retouched in conjunction with one embodiment The feature stated be suitable for all embodiments, unless it is such be characterized in it is incompatible.It is special in the spirit or essence for not departing from the disclosure In the case where sign, the disclosure can be embodied in the form of many other or otherwise be implemented, and only illustrative purpose is detailed Carefully disclose above-described embodiment of the disclosure.It should be understood that the present disclosure is not limited thereto, but allow to carry out many this fields Change and modification known to technical staff, and all such variations or modifications (weights including component of disclosed system New arrangement) it is within the scope of this disclosure.