阅读说明：本技术 语境化文本描述 (Contextualized text description ) 是由 D.维尔科斯基 C.赖斯威格 S.金 H.T.丁 于 2019-11-13 设计创作，主要内容包括：各种示例涉及分类文本的系统和方法。计算装置可以从数据库存取包括有序单词的第一集合的输入样本。计算装置可以使用单词级语言模型生成输入样本的第一语言模型特征向量,并且使用部分单词级语言模型生成输入样本的第二语言模型特征向量。计算装置可以使用目标模型、输入样本、第一语言模型特征向量、和第二语言模型特征向量生成输入样本的描述符；以及将输入样本的描述符写入到数据库。(Various examples relate to systems and methods of classifying text. The computing device may access an input sample comprising a first set of ordered words from a database. The computing device may generate a first language model feature vector of the input sample using the word-level language model and a second language model feature vector of the input sample using the partial word-level language model. The computing device may generate a descriptor of the input sample using the target model, the input sample, the first language model feature vector, and the second language model feature vector; and writing the descriptors of the input samples to the database.)

1. A computerized system for classifying text, comprising:

a database; and

at least one processor programmed to perform operations comprising:

accessing an input sample comprising a first set of ordered words from the database;

generating a first language model feature vector for the input sample using a word-level language model;

generating a second language model feature vector for the input sample using a partial word-level language model;

generating a descriptor of the input sample using a target model, the input sample, the first language model feature vector, and the second language model feature vector; and

writing descriptors of the input samples to the database.

2. The computerized system of claim 1, wherein the target model is a partial word-level model, and wherein the descriptor of the input sample describes a first partial word of the input sample.

3. The computerized system of claim 2, wherein generating the first language model feature vector comprises generating a first word-level feature vector using the word-level language model and a first word of the input sample, the first word containing the first partial word.

4. The computerized system of claim 3, wherein the first language model feature vector comprises the first word-level feature vector.

5. The computerized system of claim 3, wherein generating the first language model feature vector comprises:

generating a first word-level feature vector using the first word; and

determining the first language model feature vector based at least in part on the first word-level feature vector.

6. The computerized system of claim 1, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

7. The computerized system of claim 6, the operations further comprising generating a first partial word feature vector using the partial word-level language model and a first partial word, wherein the first word comprises the first partial word.

8. The computerized system of claim 7, wherein the partial word-level language model is a forward model, wherein the first partial word is an initial partial word of the first word, the operations further comprising selecting the first partial word feature vector as the second language model feature vector.

9. The computerized system of claim 7, the operations further comprising:

generating a second partial word feature vector using the partial word-level language model and a second partial word feature vector, wherein the first word comprises the second partial word feature vector; and

generating the second language model feature vector using the first partial word feature vector and the second partial word feature vector.

10. The computerized system of claim 9, the operations further comprising performing an attention mechanism to select the first partial word feature vector or the second partial word feature vector as the second language model feature vector.

11. The computerized system of claim 1, wherein the partial word-level language model is a sub-word-level model, and the operations further comprise generating a third language model feature vector for the input sample using a character-level model, wherein the generation of descriptors for the input sample is also based on the third language model feature vector.

12. The computerized system of claim 1, the operations further comprising:

generating a partial word-level language model forward feature vector using a first partial word of the input sample, a forward layer of the partial word-level language model, and at least one partial word of the input sample preceding the first partial word; and

generating a partial word-level language model inverse feature vector using a first partial word of the input sample, a backward layer of the partial word-level language model, and at least one partial word of the input sample that follows the first partial word.

13. The computerized system of claim 12, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

14. A method of classifying text, the method comprising:

accessing, by a computing device, an input sample comprising a first set of ordered words from a database;

generating, by the computing device, a first language model feature vector for the input sample using a word-level language model;

generating, by the computing device, a second language model feature vector for the input sample using a partial word-level language model;

generating, by the computing device, a descriptor of the input sample using a target model, the input sample, the first language model feature vector, and the second language model feature vector; and

writing descriptors of the input samples to the database.

15. The method of claim 14, wherein the target model is a partial word-level language model, and wherein the descriptor of the input sample describes a first partial word of the input sample.

16. The method of claim 15, wherein generating the first language model feature vector comprises generating a first word-level feature vector using the word-level language model and a first word of the input sample, the first word containing the first partial word.

17. The method of claim 16, wherein the first language model feature vector comprises the first word-level feature vector.

18. The method of claim 16, wherein generating the first language model feature vector comprises:

generating a first word-level feature vector using the first word; and

determining the first language model feature vector based at least in part on the first word-level feature vector.

19. The method of claim 14, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

20. A non-transitory machine-readable medium comprising instructions thereon, which when executed by at least one processor, cause the at least one processor to perform operations comprising:

accessing an input sample comprising a first set of ordered words from a database;

generating a first language model feature vector for the input sample using a word-level language model;

generating a second language model feature vector for the input sample using a partial word-level language model;

generating a descriptor of the input sample using a target model, the input sample, the first language model feature vector, and the second language model feature vector; and

writing descriptors of the input samples to the database.

Technical Field

This document relates generally to methods and systems for use with computer devices including networked computing devices. More particularly, the present document relates to a system and method of generating descriptors of text samples.

Background

Natural language processing is an important way for human users to interact with computing devices. Traditionally, computing devices have been configured to receive input from human users in specialized formats and grammars. Natural language processing allows a computing device to understand words arranged according to the grammar of a human language. However, the nuances of human language make natural language processing a technically challenging proposition.

Drawings

The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings.

FIG. 1 is a schematic diagram of one example of an environment for performing natural language processing tasks.

FIG. 2 is a schematic diagram of another example of an environment for performing natural language processing tasks.

FIG. 3 is a schematic diagram of yet another example of an environment for performing natural language processing tasks.

FIG. 4 is a schematic diagram of yet another example of an environment for performing natural language processing tasks.

FIG. 5 is a flow diagram illustrating one example of a process flow that may be executed by a computing device to perform a natural language processing task.

FIG. 6 is a flow diagram illustrating one example of a process flow that may be performed by a computing device to train a language model.

FIG. 7 is a schematic diagram of one example of an environment for training the target model of FIG. 1.

FIG. 8 is a flow diagram illustrating one example of a process flow that may be executed by a computing device to train a target model.

FIG. 9 is a block diagram of one example of a software architecture of a computing device.

Fig. 10 is a block diagram of a machine in the example form of a computer system within which instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein.

Detailed Description

The following description includes illustrative systems, methods, techniques, instruction sequences, and computer program products that implement the illustrative embodiments. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of embodiments of the inventive subject matter. It will be apparent, however, to one skilled in the art that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques have not been shown in detail.

Various examples described herein are to improve natural language processing using a target model and multiple language models having different language models operating at different levels (e.g., word level, sub-word level, and/or character level). Natural language processing tasks (such as key value extraction, summarization, sentiment analysis, etc.) involve the use of a computing device to determine the meaning of one or more words arranged according to a human language.

A natural language processing task operating on input samples operates on input samples that contain ordered sets of words. The word may be a selection of characters separated by spaces. Consider the following samples:

sample 1:

“She made a deposit to her bank account”。

sample 1 includes the words "She", "made", etc. Each word includes one or more characters separated by spaces. For example, the word "made" contains a collection of characters separated by a space from the previous word (She) in the sample and the next word (a) of the sample.

In human language, the meaning of one or more words often depends on the context in which the words are used. The following samples were considered in conjunction with sample 1 above:

sample 2:

“He sat down on the bank of the stream”。

both sample 1 and sample 2 comprise ordered sets of words. The word "bank" has different meanings in sample 1 and sample 2. In sample 1, the word "bank" refers to a financial institution. In sample 2, the word "bank" refers to a land area near a river or lake. The difference in meaning is discernable from other words in the sample. In the examples of samples 1 and 2, each word contains only alphabetic characters, but in other examples the character string may also include numbers and/or other characters that are appended to or in addition to the alphabetic characters.

When performing natural language processing tasks, the computing device may interpret the context of the words by using the target model in conjunction with the language model. The target model is used to perform natural language processing tasks on input samples containing ordered sets of words, such as the example sample 1 and sample 2 described above. Words may include letters, numbers, symbols, and the like. A language model is another model for generating one or more feature vectors based on input samples. The one or more feature vectors indicate a context of the input sample. The target model is used to perform a natural language processing task based on the input sample and one or more feature vectors.

In some examples, the language model is or includes a neural network or other layered model. For example, the language model may be implemented in multiple layers. Each layer may include a collection of nodes. The input samples are provided to nodes of a first layer, which generates first layer outputs. The nodes of a subsequent layer receive input from the nodes of a previous layer and provide output to the nodes of the next layer (if any). The output of the language model is given by the output of the node at the last layer. The output of a given layer forms a feature vector. For example, the output of a node at the first layer may form a first feature vector. The outputs of the nodes at the second layer may form a second eigenvector, and so on. These feature vectors may be provided as input to the target model, as described herein.

In some example embodiments, the language model is implemented as an auto-encoder. An auto-encoder language model is trained to reconstruct the input samples. For example, the auto-encoder language model may be trained to predict the value of a subject word based on other words or characters of the input sample. For example, referring to sample 1 above, the auto-encoder language model may be programmed to return the word "bank" when input samples containing other words of sample 1 are provided in the indicated order.

The models (including the target model and the language model) may operate on different levels, where the levels of the model describe portions of the sample indicated by the model output. For example, the model may be a word-level model, a sub-word-level model, or a character-level model. The word-level model provides an output that describes words from the input sample. For example, the word-level language model of sample 1 received above may be programmed to return the word "bank" (or other words from sample 1) when some or all of the other portions of sample 1 are provided. The word-level object model for sample 1 above may be programmed to return a descriptor for the word "bank" when some or all of the other portions of sample 1 are provided.

The partial word-level model provides an output that describes partial words (e.g., less than all of the words). One type of partial word-level model is a character-level model. The character-level model provides an output that describes characters from the input sample. For example, the character-level language model receiving sample 1 may be programmed to return the character "b" from the word "bank" when some or all of the other characters of sample 1 are provided. The character-level language model that receives sample 1 may be programmed to return a descriptor for the letter "b" based on some or all of the other characters of sample 1. Another type of partial word-level model is a sub-word-level model. The sub-word level model provides an output that describes sub-words (e.g., portions of a word that contain more than one character). For example, the sub-word level language model that receives sample 1 may be programmed to return the sub-word "ban" taking into account other sub-words of sample 1. The sub-word level object model that receives sample 1 may be programmed to return descriptors for the sub-word "ban" taking into account other sub-words of sample 1.

In many model systems, the language model and the target model are implemented at the same level (e.g., word level, sub-word level, or character level). This may simplify the operation of the system, as the feature vectors generated using the language model describe the same level used by the target model. For example, the word-level language model generates word-level feature vectors, which can be provided to various layers of the word-level target model.

In some examples, however, it may be advantageous for the model system to contain multiple language models operating at different levels (e.g., word level and partial word level). For example, feature vectors generated by the language model at different levels may provide different contextualized data. Considering contextualized data from different language models and different levels can improve the accuracy of the target model.

However, utilizing a language model that is implemented at a different level than the target model poses technical challenges. For example, feature vectors generated by a language model at one level (e.g., word level, sub-word level, character level) may not be directly related to the target model at another level. Thus, some examples described herein relate to a model system in which feature vectors from a language model at one level are used with a target model at another level.

In some examples, a lower level language model is used with a higher level object model. For example, the character-level language model is at a lower level than the sub-word-level language model and at a lower level than the word-level language model. The sub-word level language model is at a higher level than the character level language model and at a lower level than the word level language model.

When a lower level language model is used with a higher level object model, the lower level language model generates more than one set of feature vectors for the units considered by the object model. Consider an example in which a character-level language model is used in conjunction with a word-level object model to consider sample 1 above. In this example, the object model generates a descriptor of the word "bank". Because the language operates at the character level, however, a single set of feature vectors describing the word "bank" may not be generated. Alternatively, because operations are performed on individual characters, the character-level language model may generate one set of feature vectors for the character "b", another set of feature vectors for the character "a", and so on.

The model system is configured to aggregate feature vectors of lower level language models to generate higher level feature vectors of the target model. In some examples, the model system averages the lower-level feature vectors to generate a set of higher-level feature vectors. Referring back to the example above, the set of feature vectors for character "b" may be averaged with the set of feature vectors for other characters to generate a word-level feature vector. In another example, the model system is configured to select a set of lower-level feature vectors to use as the output of the higher-level object model. Referring here to the example above, the model system may select a set of feature vectors for one of the characters of the word "bank" to use as input for the target model.

In some examples, the set of feature vectors is selected based on a guideline of a language model. For example, a forward language model (e.g., a model that considers the context before a subject word, sub-word, or character) can use a set of feature vectors corresponding to the last component (e.g., last character, last sub-word) of the unit considered by the target model. The backward language model (e.g., a model that considers the context after the subject word, sub-word, or character) may use a set of feature vectors corresponding to a first component (e.g., a first character, a first sub-word) of the unit considered by the target model.

In some examples, the model system uses an attention mechanism to select the optimal lower-level feature vector generated by the lower-level language model. Note that the mechanism is a trained submodel that selects lower-level feature vectors from a set of lower-level feature vectors. Note that the mechanism may be trained to select lower-level feature vectors, for example, during training of the target model to maximize performance of the target model.

In some examples, a higher level language model is used with a lower level object model. When a higher level language model is used with a lower level target model, the higher level language model produces less than one set of feature vectors for the units considered by the target model. Consider an example in which a word-level language model is used in conjunction with a character-level object model to consider sample 1 above. In this example, the object model generates a descriptor from the letter "b" of the word "bank". Because the language model operates at the word level, it generates a collection of feature vectors for the word "bank," but not specifically for the character "b.

The model system may be configured to derive lower-level feature vectors of the target model from higher-level feature vectors generated by the language model. In some examples, the model system uses a set of higher-level feature vectors for each lower-level cell. For example, returning to the "bank" example, if the word-level language model generates a set of word-level feature vectors for the word "bank," the word-level feature vectors may be used by the character-level object model to generate a descriptor for each character of the word "bank. In another example, the model system is configured to include a separate sub-network trained to extract a set of lower-level feature vectors from a set of higher-level feature vectors.

Also, some model systems described herein may utilize a language model having forward and backward layers. The forward layer is trained to consider the context in the sample that occurs before the unit under consideration. The backward layer is trained to consider the context in the sample that occurs after the cell under consideration. A language model with a forward layer and a backward layer may generate multiple sets of feature vectors for each considered unit (e.g., word, sub-word, character). In these examples, the two sets of feature vectors may be combined in various suitable manners (including, for example, additional, concatenated, and/or other linear combinations).

FIG. 1 is a schematic diagram of one example of an environment 100 for performing natural language processing tasks. The environment 100 includes a computing device 102. The computing device 102 may be or include any suitable type of computing device or networked device, including, for example, a desktop computer, laptop computer, tablet computer or other computing device, server, or the like. Example hardware and software architectures to implement the computing device 102 are provided herein, such as in fig. 9 and 10. In the example of fig. 1, computing device 102 contains database 103 or is in communication with database 103. The database 103 may contain one or more tables for storing input samples 108 and/or input sample descriptors as described herein.

The computing device 102 executes the target model 104 and the language models 106A, 106B. The language models 106A, 106B include a character-level language model 106A and a word-level language model 106B. The language models 106A, 106B may be or include an auto-encoder language model. The language models 106A, 106B and the target model 104 receive input samples 108 that may be retrieved from the database 103. In the example of FIG. 1, the input sample is "the quick brown fox".

To consider the input sample 108, the character-level language model 106A executes an instance 114A, 114B, 114C, 114D, 114E, 114F, 114N for each character of the input sample 108. For example, one instance 114A is performed on the character "t", and so on. The number of instances 114A, 114B, 114C, 114D, 114E, 114F, 114N of the language model 106A performed on different input samples may be different, e.g., based on the number of characters of the input samples. Each instance 114A, 114B, 114C, 114D, 114E, 114F, 114N may have one or more layers containing any suitable number of nodes. The layers of examples 114A, 114B, 114C, 114D, 114E, 114F, 114N produce outputs that are provided to the next layer. The output of the layer may be used as a feature vector. For example, an instance 114A, 114B, 114C, 114D, 114E, 114F, 114N having one layer produces a set of feature vectors that includes a single feature vector. An instance 114A, 114B, 114C, 114D, 114E, 114F, 114N having multiple layers produces a set of feature vectors, including, for example, one feature vector per layer.

The word-level language model 106B considers the input sample 108 by each word execution instance 116A, 116B, 116C, 116D of the input sample 108. In this example, the first instance 116A corresponds to the word "the," the second instance 116B corresponds to the word "quick," and so on. The number of instances 116A, 116B, 116C, 116D of the language model 106B performed on different input samples may be different, for example based on the number of words in the input samples.

The target model 104 is trained to perform natural language processing tasks. Natural language processing tasks include tasks that determine the meaning of a word or group of words. One type of natural language processing task is key value extraction. Key value extraction involves detecting one or more words of a particular type indicated by a key in a sample. Consider an example key for "name". Key value extraction includes identifying words from one or more samples as names. Consider another example key for "invoice amount". Key value extraction includes identifying words from one or more samples, such as invoices or other business documents, that indicate the invoice amount.

Another example natural language processing task is summarization. Summarization involves determining important information from a sample and generating a truncated version of the sample. Another example natural language processing task is emotion analysis. Sentiment analysis includes analyzing a sample and determining the author's overall attitude or mood, and/or the author's attitude or mood with respect to one or more particular topics described by the sample.

In the example of FIG. 1, the target model 104 is a word-level model. The object model 104 may include multiple layers 112A, 112B, where each layer 112A, 112B includes one or more nodes. A node at a given layer 112A, 112B may receive as input the input samples 108 and/or the output of a previous layer. For example, here the first layer 112A receives as input the input samples 108. The nth layer 112N receives as input the output of the nth-1 layer. The target model 104 generates an input sample descriptor 110 that describes the cell under consideration of the input sample 108. As here, where the target model 104 is a word-level model, the units under consideration are words. For example, the input sample descriptor 110 may indicate the results of a natural language processing task (such as key values, etc.) with respect to the word under consideration.

In the example of fig. 1, the considered units of the target model 104 are from the input samples 108: the word "brown" of "The quick brown fox". The word-level language model 106B provides a set 122B of feature vectors from the instance 116C that was performed on the word "brown". A set 122B of feature vectors may be provided to each layer 112A, 112N of the target model 104. The character-level language model 106A generates a set of feature vectors from instances 114B, 114C, 114D, 114E, 114F corresponding to the characters "B", "r", "o", "w", and "n" of the word "brown". The set of these feature vectors is provided to a feature vector aggregator 120 that generates a single set of feature vectors 122A, which single set of feature vectors 122A is provided to the layers 112A, 112N of the target model 104. The target model 104 generates an input sample descriptor 110 based on the input sample 108 and the set of feature vectors from the language models 106A, 106B.

The feature vector aggregator 120 may generate a set of feature vectors 122A from the set of feature vectors generated by the instances 114B, 114C, 114D, 114E, 114F. In some examples, the feature vector aggregator 120 selects a set of feature vectors from one of the instances 114B, 114C, 114D, 114E, 114F. For example, when language model 106A is a forward model, feature vector aggregator 120 may select a set of feature vectors from instances 114F corresponding to the character "n" as set of feature vectors 122A. In this manner, the set of selected feature vectors 122A may consider context from all previous words in the input sample 108 and the characters preceding the word under consideration, the "brown". Similarly, when the language model 106A is a backward model, the feature vector aggregator 120 may select a set of feature vectors from the instances 114B corresponding to the character "B" as the set of feature vectors 122A. In this manner, the set of selected feature vectors 122A may consider context based on all following words in the input sample 108 and the characters following the word under consideration, the "brown".

In another example, the feature vector aggregator 120 generates a set of feature vectors 122A that is an average of the sets of feature vectors generated by the instances 114B, 114C, 114D, 114E, 114F. Consider an example in which the character-level language model 106A contains two layers. Thus, each instance 114B, 114C, 114D, 114E, 114F will generate a set of feature vectors that includes a first-layer feature vector and a second-layer feature vector. The set of feature vectors 122A may include a first-layer feature vector that is an average of all first-layer feature vectors of the instances 114B, 114C, 114D, 114E, 114F and a second-layer feature vector that is an average of all second-layer feature vectors of the instances 114B, 114C, 114D, 114E, 114F.

In another example, the feature vector aggregator 120 implements an attention mechanism. The attention mechanism is adjusted to select the set of feature vectors generated by one or more of the instances 114B, 114C, 114D, 114E, 114F that best fits the target model 104. In some examples, the attention mechanism generates a linear combination of all the feature vectors 114B, 114C, 114D, 114E, 114F within a word or sub-word. The linearly combined coefficients may be learned, for example, using additional hidden layers and attention mechanism context vectors. Note that the mechanistic context vector may be trained to indicate which instance 114B, 114C, 114D, 114E, 114F is most informative for a given word or sub-word. For example, the feature vectors may be transformed into a hidden representation by an additional hidden layer. The attention mechanism context vector may then be compared to all hidden representations and a similarity score may be generated for each character. The normalized weights and/or coefficients of the linear combination may be determined by applying a softmax function to the resulting scores. As described herein, additional hidden layers and attention mechanism context vectors may be trained with the remainder of the target model.

Although the feature vector aggregator 120 in this example converts the set of feature vectors from the character-level language model 106A for use at the word-level language model 104, the techniques of the feature vector aggregator 120 may also be used to convert the set of feature vectors from a lower-level language model for use at a higher-level target model. For example, the feature vector aggregator 120 may also be used to generate a single set of feature vectors from multiple sets of feature vectors generated by the sub-word level language model. FIG. 2 is a schematic diagram of another example of an environment 200 for performing natural language processing tasks.

The environment 200 of FIG. 2 is similar to the environment 100 of FIG. 1, but includes a character-level object model 204 having layers 212A, 212N. The cells under consideration of the object model 204 are then characters. In this example, the cell under consideration is the character "b" from the word "brown". The target model 204 receives the set of feature vectors 222A from the instance 114A of the character-level language model 106A that considers the character "b".

In this example, the word-level language model 106B generates a set of feature vectors (e.g., instance 116C) that correspond to the word "brown," but not specifically to the character "B" under consideration. A feature vector solution aggregator 220 is provided to generate a set of feature vectors 222B provided by the word-level language model 106B to the target model 204. The feature vector de-aggregator 220 may select the feature vector set 222B in any suitable manner. In some examples, the feature vector solution aggregator 220 may reuse the word-level set of feature vectors for each character in the alphabet. In the example of FIG. 2, this means that the feature vector solution aggregator 220 will pass through the set of feature vectors generated by instance 116C, while the target model 204 considers the character "b" (and other letters in the word "brown"). In other examples, the feature vector solution aggregator 220 may upsample (upscale), for example, a set of feature vectors 222B using a trained network to correspond to the considered characters of the target model 204.

The upsampling may be performed, for example, using a sub-network of one or more layers containing successively transposed one-dimensional convolutions, such as fractional step convolutions. The convolution may interpolate or upsample from a lower level of resolution (e.g., word level or sub-word level) to a higher level of resolution (e.g., sub-word level or character level). In some examples, to obey the forward/backward character target resolution, the convolution is decentered, for example, by shifting the output or by masking out these elements of the convolution kernel from future or past readings. This may prevent the subnetwork from using information from the future in the case of a forward model or from the past in the case of a backward model. In some examples, the sub-networks are trained jointly with the language model.

FIG. 3 is a schematic diagram of yet another example of an environment 300 for performing natural language processing tasks. The environment 300 of FIG. 3 may be similar to the environment 100 of FIG. 1, with the addition of a sub-word level language model 106C that also receives the input samples 108. To consider the input samples 108, the sub-word level language model 106C may divide the input samples 108 into sub-words. The sub-words may be determined in any suitable manner. In this example, the input sample "the quick brown fox" is divided into the sub-words "the", "qu", "ick", "br", "wn" and "fox". The sub-word level language model 106C executes an instance 117A, 117B, 117C, 117D, 117E, 117F for each sub-word of the input sample 108. For example, an instance 117A is performed on the sub-word "the"; an instance 117B is performed on the sub-word "qu", and so on. The number of instances 117A, 117B, 117C, 117D, 117E, 117F of the language model 106C that are performed on different input samples may be different, e.g., based on the number of characters of the input samples. As described herein, each instance 117A, 117B, 117C, 117D, 117E, 117F can produce a set of feature vectors.

In the example of FIG. 3, similar to FIG. 1, a word-level object model 104 is used. Thus, the feature vector aggregator 320 is used to generate a set of feature vectors 122C for use at the word-level model 104 from the plurality of sub-word-level feature vectors generated by the instances 117A, 117B, 117C, 117D, 117E, 117F. In the example of FIG. 3, the considered unit of the target model 104 is the word "brown" as shown in FIG. 1. Thus, the feature vector aggregator 320 may generate the feature vector set 122C from a set of feature vectors generated by instances 117D, 117E in the language model 106C, the instances 117D, 117E considering the sub-words "br" and "own" that make up the word "brown" under consideration.

The feature vector aggregator 320 may operate in a similar manner as the feature vector aggregator 120. For example, when the language model 106C is a forward model, the feature vector aggregator 320 may select a set of feature vectors from the instance 117F corresponding to the sub-word "own" as the set of feature vectors 122C. In this manner, the set of selected feature vectors 122C may consider context from all previous words in the input sample 108 and the sub-word characters preceding the word under consideration, the "brown". Similarly, when the language model 106C is a backward model, the feature vector aggregator 320 may select a set of feature vectors from the instance 117D corresponding to the sub-word "br" as the set of feature vectors 122C. In this manner, the set of selected feature vectors 122C may consider context based on all following words in the input sample 108 and sub-word characters following the word under consideration, the "brown". In another example, the feature vector aggregator 320 generates a set of feature vectors 122C, which is an average of the sets of feature vectors generated by the instances 117D, 117E.

In the example of FIG. 3, the target model 104 is a word-level model. In some examples, however,

the target model in the arrangement of fig. 3 may be a sub-word level model or a character level model. In an example where the target model is a sub-word level model, a feature vector de-aggregator (similar to de-aggregator 220) may be used to generate the set of feature vectors 122B of the language model 106B. The feature vector aggregator 320 may be omitted. In an example where the target model is a character-level model, the feature vector aggregator 120 may be omitted. The word-level language model 106B and the sub-word-level language model 106C may utilize a feature vector solution aggregator, such as the feature vector solution aggregator 220 described herein.

FIG. 4 is a schematic illustration of yet another example of an environment 400 for performing natural language processing tasks. In the example of FIG. 4, language models 406A, 406B, 406C execute forward layers 450, 454, 458 and backward layers 452, 456, 460. For example, character-level language model 406A includes a forward layer 450 and a backward layer 452. Word-level language model 406B includes a forward layer 458 and a backward layer 460. The sub-word level language model 406C includes a forward layer 454 and a backward layer 456. The forward layers 450, 454, 458 may contain instances of the respective models 406A, 406B, 406C that take into account the context in front of the element under consideration. The backward layers 452, 456, 460 may contain instances of the respective models 406A, 406B, 406C that take into account the context behind the considered element. Thus, each cell of the input sample 108 may be considered at the language model by a forward instance and a backward instance. The feature vectors from the forward and backward instances may be combined, for example, by concatenating the feature vectors, adding the feature vectors, using trainable linear combinations, or by other suitable methods. Feature vectors from a combination of corresponding forward and reverse instances may be used to generate the feature vector sets 122A, 122B, 122C as described herein. Although fig. 4 shows a similar arrangement as fig. 3 with forward and reverse layers of a language model, it will be understood that the forward and reverse language model layers may also be used in any of the arrangements described with respect to fig. 1 and 2. FIG. 5 is a flow diagram illustrating one example of a process flow 500 that may be executed by a computing device, such as computing device 102, to perform a natural language processing task.

At operation 502, the computing device generates a first set of feature vectors with a first language model. As described herein, the first set of feature vectors may include one or more feature vectors. At operation 504, the computing device generates a second set of feature vectors with a second language model. In some examples, the second language model is at a different level than the first language model. The second set of feature vectors may include one or more feature vectors. At optional operation 506, the computing device generates a third set of feature vectors with a third language model. The third language model is at a different level than the first language model or the second language model. The third set of feature vectors may include one or more feature vectors. At operation 508, the computing device generates an input sample descriptor with the set of target models and feature vectors.

FIG. 6 is a flow diagram illustrating one example of a process flow 600 that may be performed by a computing device (such as computing device 102 of FIG. 1) to train a language model. The process flow 600 may be used to train any of the language models 106A, 106B, 106N. At operation 602, the computing device accesses language model training samples from a language model training corpus. In some examples, the language model training corpus is unlabeled. For example, samples of a language model training corpus and/or words thereof may not be associated with metadata describing the samples and/or words. In some examples, a computing device generates language model training samples. For example, the computing device may select the next n units (e.g., characters, sub-words, words) from the language model training corpus, where the next n units constitute the language model training samples. In some examples, the training samples are stored at tables of database 103 and accessed therefrom.

At operation 604, the computing device provides training samples to the language model. The language model processes the training samples to generate the next unit of modeling after the training samples. The next element modeled may be the next word, the next character, the next sub-word, etc., depending on the level of the language model being trained. At operation 606, the computing device receives the modeled next unit and compares it to the actual next unit at the training corpus.

At operation 608, the computing device provides feedback to the language model based on the comparison of the modeled next unit to the actual next unit. In some examples, the feedback includes a modification to the language model. For example, the computing device may modify one or more layers or nodes at the language model. At operation 610, the computing device determines whether to perform more training. Additional training may be performed, for example, if the language model has an error rate greater than a threshold. If there is no additional training to perform, the process may be completed at operation 612. If more training is to be performed, the training service accesses the next language model sample at operation 602 and continues to utilize the next language model sample as described herein.

FIG. 7 is a schematic diagram illustrating one example of an environment 700 for training the target model 104 of FIG. 1. In environment 700, computing device 102 executes the model arrangement of FIG. 1, which contains character-level language model 106A and word-level language model 106B. The target model training service 704 trains the target models, for example, using a training corpus 706.

The target model training service 704 is configured to train the target model 104 in any suitable manner (e.g., based on natural language processing tasks to be performed by the target model 104). FIG. 8 is a flow diagram illustrating one example of a process flow 800 that may be performed by the computing device 102 (e.g., its target model training service 704) to train the target model 104. Although the process flow 800 is described with respect to FIG. 7, the process flow 800 may also be used to train the target model described with respect to any arrangement herein.

At operation 802, the target model training service 704 accesses target model training samples from the target model training corpus 706. In some examples, the target model training corpus 706 contains labeled training samples. For example, the labeled training samples include or are associated with labels that indicate training descriptors of the training samples. The training descriptors describe the correct results of the relevant natural language processing tasks of the target model 104 being trained. For example, when the target model 104 is being trained to perform key-value extraction, some or all of the training samples of the target model training corpus 706 are associated with labels (if any) that indicate keys for various character strings of the training samples. In examples where the target model 104 is being trained to perform summarization, the training samples may be associated with labels that indicate the summarization of the respective training samples. In examples where the target model 104 is being trained to perform emotion analysis, the training samples may be associated with labels that indicate emotional associations with one or more strings in the training samples, and/or the entire training sample.

At operation 804, the target model training service 704 provides target model training samples to the language models 106A, 106B. Additional language models, such as language model 106, may also be provided with a target model training loop if such language models exist. At operation 806, the set of feature vectors 122A, 122B is provided to the layers 112A, 112N of the target model 104 as described herein.

At operation 808, the target model training service 704 receives training sample descriptors from the target models 104. The training sample descriptors are the output of the desired natural language processing task as performed on the target model training samples. For example, when the natural language processing task is key value extraction, the training sample descriptor may be a set of one or more keys of a string that describes the target model training sample. In other examples where the target model 104 is used to perform other natural language processing tasks, the training sample descriptor may have other values and/or forms as described herein.

At operation 810, the target model training service 704 provides feedback to the target model 104 based on a comparison of the training sample descriptors and the labels associated with the target model training samples. In some examples, the feedback includes a modification to the target model 104. For example, the training service 704 may modify characteristics of one or more nodes of the layers 112A, 112N of the target model. Example node characteristics that may be modified include, for example, weights applied to the corresponding feature vectors. The training service 704 may utilize appropriate back propagation techniques to convert the delta between the training sample descriptor and the target model training sample label into a specific modification to the node.

At operation 812, the target model training service 704 determines whether more training is to be performed. Additional training may be performed, for example, if the target model 104 has an error rate greater than a threshold. If there is no additional training to perform, the process may be completed at operation 814. If more training is to be performed, the training service accesses the next target model sample at operation 802 and continues to utilize the next target model sample as described herein.

Example (c):

example 1 is a computerized system to classify text, comprising: a database; and at least one processor programmed to perform operations comprising: accessing an input sample comprising a first set of ordered words from a database; generating a first language model feature vector of the input sample using the word-level language model; generating a second language model feature vector of the input sample using the partial word level language model; generating descriptors of the input sample using the target model, the input sample, the first language model feature vectors, and the second language model feature vectors; and writing the descriptors of the input samples to the database.

In example 2, the subject matter of example 1 optionally includes, wherein the target model is a partial word-level model, and wherein the descriptor of the input sample describes a first partial word of the input sample.

In example 3, the subject matter of example 2 optionally includes wherein generating the first language model feature vector includes generating a first word-level feature vector using a word-level language model and a first word of the input sample, the first word including the first partial word.

In example 4, the subject matter of example 3 optionally includes wherein the first language model feature vector comprises a first word-level feature vector.

In example 5, the subject matter of any one or more of examples 3-4 optionally includes, wherein generating the first language model feature vector comprises: generating a first word-level feature vector using the first word; and determining a first language model feature vector based at least in part on the first word-level feature vector.

In example 6, the subject matter of any one or more of examples 1-5 optionally includes, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

In example 7, the subject matter of example 6 optionally includes generating a first partial word feature vector using the partial word level language model and the first partial word, wherein the first word comprises the first partial word.

In example 8, the subject matter of example 7 optionally includes wherein the partial word level language model is a forward model, wherein the first partial word is an initial partial word of the first word, the operations further comprising selecting the first partial word feature vector as the second language model feature vector.

In example 9, the subject matter of any one or more of examples 7-8 optionally includes, the operations further comprising: generating a second partial word feature vector using the partial word-level language model and the second partial word feature vector, wherein the first word includes the second partial word feature vector; and generating a second language model feature vector using the first partial word feature vector and the second partial word feature vector.

In example 10, the subject matter of example 9 optionally includes performing an attention mechanism to select the first partial word feature vector or the second partial word feature vector as the second language model feature vector.

In example 11, the subject matter of any one or more of examples 1-10 optionally includes, wherein the partial word-level language model is a sub-word-level model, and the operations further comprise generating a third language model feature vector for the input sample using the character-level model, wherein generating the descriptor for the input sample is also based on the third language model feature vector.

In example 12, the subject matter of any one or more of examples 1-11 optionally includes, the operations further comprising: generating a partial word-level language model forward feature vector using a first partial word of the input sample, a forward layer of the partial word-level language model, and at least one partial word of the input sample preceding the first partial word; and generating a partial word-level language model inverse feature vector using the first partial word of the input sample, a backward layer of the partial word-level language model, and at least one partial word of the input sample that follows the first partial word.

In example 13, the subject matter of example 12 optionally includes, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

Example 14 is a method of classifying text, the method comprising: accessing, by a computing device and from a database, an input sample comprising a first set of ordered words; generating, by a computing device, a first language model feature vector for an input sample using a word-level language model; generating, by the computing device, a second language model feature vector for the input sample using the partial word-level language model; generating, by a computing device, a descriptor of an input sample using a target model, the input sample, a first language model feature vector, and a second language model feature vector; and writing the descriptors of the input samples to the database.

In example 15, the subject matter of example 14 optionally includes, wherein the target model is a partial word-level language model, and wherein the descriptor of the input sample describes a first partial word of the input sample.

In example 16, the subject matter of example 15 optionally includes, wherein generating the first language model feature vector includes generating the first word-level feature vector using a word-level language model and a first word of the input sample, the first word including the first partial word.

In example 17, the subject matter of example 16 optionally includes the first language model feature vector comprising a first word-level feature vector.

In example 18, the subject matter of any one or more of examples 16-17 optionally includes, wherein generating the first language model feature vector comprises: generating a first word-level feature vector using the first word; and determining a first language model feature vector based at least in part on the first word-level feature vector.

In example 19, the subject matter of any one or more of examples 14-18 optionally includes, wherein the target model is a word-level model, and wherein the descriptor of the input sample describes a first word of the input sample.

Example 20 is a non-transitory computer-readable medium comprising instructions thereon, which when executed by at least one processor, cause the at least one processor to perform operations comprising: accessing an input sample comprising a first set of ordered words from a database; generating a first language model feature vector of the input sample using the word-level language model; generating a second language model feature vector of the input sample using the partial word level language model; generating descriptors of the input samples using the target model, the input samples, the first language model feature vectors, and the second language model feature vectors; and writing the descriptors of the input samples to the database.

Fig. 9 is a block diagram 900 of one example of a software architecture 902 of a computing device. The architecture 902 may be used, for example, in conjunction with various hardware architectures as described herein. Fig. 9 is only a non-limiting example of a software architecture, and many other architectures may be implemented to facilitate the functionality described herein. A representative hardware layer 904 is shown and may represent, for example, any of the computing devices referenced above. In some examples, the hardware layer 904 is implemented according to the architecture of the computing system 1000 of fig. 10.

The representative hardware layer 904 includes one or more processing units 906 having associated executable instructions 908. Executable instructions 908 represent executable instructions of the software architecture 902, including implementing the methods, modules, subsystems, components, and the like described herein, and the hardware layer may also include memory and/or storage modules 910 that also have executable instructions 908. The hardware layer 904 may also include other hardware, as indicated by other hardware 912, which represents other hardware of the hardware layer 904, such as that illustrated by portions of the computer system 1000.

In the example architecture of FIG. 9, the software architecture 902 may be conceptualized as a stack of layers, where each layer provides specific functionality. For example, software architecture 902 may contain layers such as: an operating system 914, a library 916, a framework/middleware 918, applications 920, and a presentation layer 944. Operationally, the application 920 and/or other components within the layers may invoke an Application Program Interface (API) call 924 through the software stack and access a response, return value, etc. as illustrated by message 926 in response to the API call 924. The layers shown are representative in nature, but not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide the framework/middleware layer 918, while others may provide such a layer. Other software architectures may contain additional or different layers.

The operating system 914 may manage hardware resources and provide common services. The operating system 914 may include, for example, a kernel 928, services 930, and drivers 932. The kernel 928 may act as a side abstraction layer between a hardware layer and other software layers. For example, the kernel 928 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and the like. The services 930 may provide other common services for other software layers. In some examples, services 930 include interrupt services. Interrupt servicing may detect receipt of an interrupt and in response cause the architecture 902 to halt its current processing and execute an Interrupt Service Routine (ISR) when the interrupt is accessed.

The driver 932 may be responsible for controlling or interfacing with the underlying hardware. For example, drivers 932 may include a display driver, a camera driver,Drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers),Drivers, NFC drivers, audio drivers, power management drivers, etc., depending on the hardware configuration.

The library 916 can provide a common infrastructure that can be utilized by the application 920, and/or other components, and/or other layers. The libraries 916 typically provide functionality that allows other software modules to perform tasks in an easier manner than directly interfacing with underlying operating system 914 functionality (e.g., the kernel 928, services 930, and/or drivers 932). The library 916 may contain a system library 934 (e.g., a C-standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, and the like. Further, the libraries 916 may contain API libraries 936, such as media libraries (e.g., libraries that support representing and manipulating various media formats such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., OpenGL frameworks that may be used to render 2D and 14D with graphical content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functions), and so forth. The library 916 may also include a variety of other libraries 938 to provide many other APIs to the application 920 and other software components/modules. In some examples, the library 916 may provide one or more APIs served by the message-oriented middleware.

Framework 918 (also sometimes referred to as middleware) can provide a higher-level common infrastructure (ifrasstructure) that can be utilized by applications 920, and/or other software components/modules. For example, the framework 918 may provide various Graphical User Interface (GUI) functionality, advanced resource management, high-level location services, and the like. Framework 918 can provide a wide variety of other APIs that can be utilized by applications 920, and/or other software components/modules, some of which can be specific to a particular operating system or platform.

The applications 920 include built-in applications 940 and/or third-party applications 942. Examples of representative built-in applications 940 may include, but are not limited to, a contacts application, a browser application, a book reader application, a positioning application, a media application, a messaging application, and/or a gaming application. The third-party applications 942 may include any built-in application 940, as well as a wide variety of other applications. In a specific example, the third-party application 942 (e.g., used Android by an entity other than a platform-specific vendor)^TMOr iOS^TMApplications developed by Software Development Kit (SDK) may be in a mobile operating system (such as iOS)^TM、Android^TM、Phone or other mobile computing device operating system). In this example, the third party application 942 may cause an API call 924 provided by a mobile operating system, such as operating system 914, to facilitate the functionality described herein.

The applications 920 may utilize built-in operating system functions (e.g., kernel 928, services 930, and/or drivers 932), libraries (e.g., system 934, API 936, and other libraries 938), framework/middleware 918 to create a user interface for user interaction with the system. Alternatively or additionally, in some systems, interaction with a user may occur through a presentation layer (such as presentation layer 944). In these systems, the application/module "logic" may be separated from aspects of the application/module that interact with the user.

Some software architectures utilize virtual machines. In the example of fig. 9, this is illustrated by virtual machine 948. Virtual machines create a software environment that applications/modules can execute as if they were executing on a hardware computing device. Virtual machine 948 is hosted by a host operating system (operating system 914) and typically (although not always) has a virtual machine monitor 946 that manages the operation of virtual machine 948 and the interface with the host operating system (i.e., operating system 914). The software architecture executes within virtual machine 948, such as operating system 950, libraries 952, framework/middleware 954, applications 956, and presentation layer 958. These layers of the software architecture executing within the virtual machine 948 may be the same as or may be different from the corresponding layers previously described.

Module, component, and logic

Certain embodiments are described herein as comprising logic or multiple components, modules, or mechanisms. The modules may constitute software modules (e.g., (1) code embodied on a non-transitory machine-readable medium or (2) code embodied in a transmission signal) or hardware-implemented modules. A hardware-implemented module is a tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client, or server computer system), or one or more processors, may be configured by software (e.g., an application or application portion) as hardware-implemented modules that operate to perform certain operations described herein.

In various embodiments, hardware-implemented modules may be implemented mechanically or electronically. For example, a hardware-implemented module may comprise special-purpose circuitry or logic that is permanently configured (e.g., configured as a special-purpose processor, such as a Field Programmable Gate Array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module may also include programmable logic or circuitry (e.g., embodied within a general-purpose processor or another programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to mechanically implement a hardware-implemented module as a dedicated and permanently configured circuit or as a temporarily configured circuit (e.g., as configured by software) may be driven by cost and time considerations.

Thus, the term "hardware-implemented module" should be considered to encompass a tangible entity, be it a physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) entity, that performs in some manner the operations and/or performs some of the operations described herein. In view of embodiments in which the hardware-implemented modules are temporarily configured (e.g., programmed), each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where a hardware-implemented module includes a general-purpose processor configured using software, the general-purpose processor may be configured at different times as corresponding different hardware-implemented modules. The software may configure the processor accordingly, e.g., to constitute a particular hardware-implemented module at one instance in time and to constitute a different hardware-implemented module at a different instance in time.

A hardware-implemented module may provide information to, or receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be considered communicatively coupled. In case a plurality of such hardware implemented modules are present at the same time, the communication may be realized by signal transmission, e.g. by suitable circuits and buses connecting the hardware implemented modules. In embodiments where multiple hardware-implemented modules are configured or instantiated at different times, communication between such hardware-implemented modules may be accomplished, for example, by the storage and retrieval of information in a memory structure to which the multiple hardware-implemented modules have access. For example, a hardware-implemented module may perform an operation and store the output of the operation in a memory device communicatively coupled thereto. Other hardware-implemented modules may then later access the memory device to retrieve and process the stored output. A hardware-implemented module may also initiate communication with an input or output device and may operate on a resource (e.g., a set of information).

Various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that are operable to perform one or more operations or functions. In some example embodiments, the modules referred to herein may comprise processor-implemented modules.

Similarly, the methods described herein may be implemented at least in part by a processor. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. The performance of certain operations may be distributed among one or more processors, not only residing within a single machine, but also being deployable across multiple machines. In some example embodiments, one or more processors may be located at a single location (e.g., within a home environment, office environment, or server farm), while in other embodiments, processors may be distributed across multiple locations.

The one or more processors may also be operable to support the performance of related operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations may be performed by a set of computers (as an example of machines including processors), which may be accessed via a network (e.g., the internet) and via one or more appropriate interfaces (e.g., APIs).

Electronic device and system

Example embodiments may be implemented in digital electronic circuitry, or in computer hardware, firmware, or software, or in combinations of them. Example embodiments may be implemented using a computer program product, such as a computer program tangibly embodied in an information carrier, e.g., in a machine-readable medium, for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers.

A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

In an example embodiment, operations may be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Method operations may also be performed by, and apparatus of example embodiments may be implemented as, special purpose logic circuitry, e.g., an FPGA or an ASIC.

A computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments where a programmable computing system is deployed, it will be understood that both hardware and software architectures are considered on average. In particular, it will be appreciated that the choice of whether to implement certain functions in permanently configured hardware (e.g., an ASIC), or in temporarily configured hardware (e.g., a combination of software and a programmable processor), or in a combination of permanently configured hardware and temporarily configured hardware, may be a design choice. In various example embodiments, the following lists hardware (e.g., machine) and software architectures that may be deployed.

Example machine architecture and machine-readable media

Fig. 10 is a block diagram of a machine in the example form of a computer system 1000 within which instructions 1024 may be executed to cause the machine to perform any one or more of the methodologies discussed herein. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client computer in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a Personal Computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a Web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include a set of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example computer system 1000 includes a processor 1002 (e.g., a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) or both), a main memory 1004, and a static memory 1006, which communicate with each other via a bus 1008. The computer system 1000 may also include a video display unit 1010, such as a Liquid Crystal Display (LCD) or a Cathode Ray Tube (CRT). The computer system 1000 also includes an alphanumeric input device 1012 (e.g., a keyboard or touch-sensitive display screen), a User Interface (UI) navigation (or cursor control) device 1014 (e.g., a mouse), a disk drive device 1016, a signal generation device 1018 (e.g., a speaker), and a network interface device 1020.

Machine readable medium

The disk drive unit 1016 includes a machine-readable medium 1022 on which is stored one or more sets of data structures and instructions 1024 (e.g., software) embodied in or utilized by any one or more of the methods or functions described herein. The instructions 1024 may also reside, completely or at least partially, within the main memory 1004 and/or within the processor 1002 during execution thereof by the computer system 1000, the main memory 1004 and the processor 1002 also constituting machine-readable media 1022.

While the machine-readable medium 1022 is shown in an example embodiment to be a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more instructions 1024 or data structures. The term "machine-readable medium" shall also be taken to include any tangible medium that is capable of: store, encode, or carry instructions 1024 for execution by the machine and to cause the machine to perform any one or more of the methodologies of the present disclosure, or be capable of storing, encoding, or carrying data structures used by such instructions 1024 or associated with such instructions 1024. The term "machine-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories and optical and magnetic media. Specific examples of the machine-readable medium 1022 include: non-volatile memory, including, as examples, semiconductor memory devices (such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices); magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

Transmission medium

The instructions 1024 may also be transmitted or received over a communication network 1026 using a transmission medium. The instructions 1024 may be sent using the network interface device 1020 and any one of a number of well-known transfer protocols (e.g., HTTP). Examples of communication networks include a Local Area Network (LAN), a Wide Area Network (WAN), the internet, a mobile telephone network, a Plain Old Telephone (POTS) network, and a wireless data network (e.g., WiFi and WiMax networks). The term "transmission medium" shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions 1024 for execution by the machine, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

Although embodiments have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The accompanying drawings that form a part hereof show by way of illustration, and not of limitation, specific embodiments in which the subject matter may be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. The detailed description is, therefore, not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

Such embodiments of the inventive subject matter may be referred to herein, individually and/or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Thus, although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description.