Method, device and storage medium for generating field annotation and understanding character string

文档序号：1875848 发布日期：2021-11-23 浏览：16次中文

阅读说明：本技术 一种字段注释生成、字符串理解方法、设备及存储介质 (Method, device and storage medium for generating field annotation and understanding character string ) 是由郭立帆徐阆平于 2020-05-19 设计创作，主要内容包括：本申请实施例提供一种字段注释生成、字符串理解方法、设备及存储介质。在本申请实施例中,对于缺失字段注释的字段名,可确定其中包含的英文缩写字符串,并对英文缩写字符串进行还原,以将字段名中的英文缩写还原为英文全拼,在此基础上,可对字段名进行英文翻译,以生成字段名的字段注释。据此,本申请实施例中,无需在依赖人工方式生成字段注释,可有效提高字段注释的生成效率,而且,通过对字段名中英文缩写的准确还原,可保证生成的字段注释的准确性。(The embodiment of the application provides a field annotation generation method, a character string understanding method, a device and a storage medium. In the embodiment of the application, for the field names of the missing field comments, the english abbreviation character strings contained in the field names can be determined, and the english abbreviation character strings are restored to restore the english abbreviations in the field names to english spells, on the basis of which the field names can be translated in english to generate the field comments of the field names. Therefore, in the embodiment of the application, the field annotation does not need to be generated manually, the generation efficiency of the field annotation can be effectively improved, and the accuracy of the generated field annotation can be ensured by accurately restoring the Chinese and English abbreviations of the field names.)

1. A field comment generation method, comprising:

acquiring a field name to be processed;

determining English abbreviation character strings contained in the field names;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the field name based on the English spelling corresponding to the English abbreviation character string to generate a field annotation of the field name.

2. The method of claim 1, wherein determining the english spellings corresponding to the english abbreviation string based on the mapping relationship between the english abbreviation and the english spellings comprises:

determining at least one candidate word matched with the maximum common factor sequence from an English word library by taking the English abbreviation character string as the maximum common factor sequence;

calculating the probability that the at least one candidate word is respectively used as the English spelling of the English abbreviation character string based on the mapping relation between the English abbreviation and the English spelling;

and taking the candidate words with the probability meeting the preset requirement as English full spellings corresponding to the English abbreviation character strings.

3. The method of claim 2, wherein the calculating the probability that the at least one candidate word is each an english spell of the english abbreviation string based on a mapping relationship between the english abbreviation and the english spell comprises:

inputting the English abbreviation character string into an English abbreviation prediction model; calculating the probability of the at least one candidate word abbreviated as the English abbreviated character string in the English abbreviated prediction model based on the mapping relation between the English abbreviated and the English full spelling;

and calculating the probability that each candidate word is used as the English spelling of the English abbreviated character string according to the probability that the at least one candidate word is abbreviated as the English abbreviated character string and output by the English abbreviated prediction model based on Bayesian hypothesis.

4. The method of claim 3, wherein the acronym prediction model employs a seq2seq model.

5. The method of claim 3, wherein before entering the abbreviated English character string into the abbreviated English prediction model, the method further comprises:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

and inputting the labeled sample data set into the English abbreviation prediction model so that the English abbreviation prediction model can learn the mapping relation between the English abbreviation and the English spelling.

6. The method of claim 5, wherein said labeling the sample data set for correspondence between sample English words and sample English abbreviations comprises:

coding the sample English abbreviation to obtain a coding sequence of the sample English abbreviation, wherein the coding sequence is used for representing a common factor between the sample English abbreviation and a sample English word corresponding to the sample English abbreviation;

and establishing a corresponding relation between a coding sequence and the sample English word so that the English abbreviation prediction model can learn the mapping relation between the coding sequence and the English full spelling.

7. The method of claim 6, wherein the calculating the probability that the at least one candidate word is abbreviated as the English abbreviation character string based on the mapping relationship between the English abbreviation and the English spelling comprises:

and calculating the probability of the at least one candidate word being abbreviated as the coding sequence corresponding to the English abbreviated character string based on the mapping relation between the coding sequence and the English full spelling as the probability of the at least one candidate word being abbreviated as the English abbreviated character string.

8. The method of claim 2, wherein determining at least one candidate word from the english word library that matches the greatest common factor sequence with the english abbreviated character string as the greatest common factor sequence comprises:

determining a target industry field where the field name is located;

and determining at least one candidate word matched with the maximum common factor sequence from an English word library corresponding to the target industry field by taking the English abbreviation character string as the maximum common factor sequence.

9. The method of claim 3, wherein inputting the abbreviated English character string into an abbreviated English prediction model comprises:

determining a target industry field where the field name is located;

inputting an English abbreviation prediction model into the English abbreviation character string and the target industry field;

the calculating the probability that the at least one candidate word abbreviation is the English abbreviation character string based on the mapping relation between the English abbreviation and the English full spelling comprises the following steps:

and calculating the probability of the at least one candidate word abbreviation being the English abbreviation character string based on the mapping relation between the English abbreviation and the English full spelling in the target industry field.

10. The method of claim 1, wherein the determining the abbreviated string of english contained in the field name comprises:

if the field name contains a separation character, dividing the field name into a plurality of character segments according to the separation character;

and determining a character segment which does not belong to the English word in the plurality of character segments as the English abbreviation character string.

11. The method of claim 1, further comprising:

supplementing the field annotation corresponding to the field name to a database where the field name is located; or

And constructing an association relation between the field names and the field annotations in the database based on the field annotations corresponding to the field names and the field annotations corresponding to other field names in the database where the field names are located.

12. The method of claim 2, wherein said determining at least one candidate word matching said maximal commonality sequence from the english abbreviation string as the maximal commonality sequence further comprises:

identifying the English abbreviated character string by using the English abbreviated dictionary, and if the English abbreviated character string is determined to be in the English abbreviated dictionary, determining the English full spelling corresponding to the English abbreviated character string according to the English abbreviated dictionary;

and if the English abbreviated character string is determined not to exist in the English abbreviated dictionary, the operation of determining at least one candidate word matched with the maximum common factor sequence from the English word library by taking the English abbreviated character string as the maximum common factor sequence is executed.

13. A computing device comprising a memory and a processor;

the memory is to store one or more computer instructions;

the processor is coupled with the memory for executing the one or more computer instructions for:

acquiring a field name to be processed;

determining English abbreviation character strings contained in the field names;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the field name based on the English spelling corresponding to the English abbreviation character string to generate a field annotation of the field name.

14. The device of claim 13, wherein the processor, when determining the english spellings corresponding to the english abbreviation string based on a mapping between the english abbreviation and the english spellings, is configured to:

and taking the candidate words with the probability meeting the preset requirement as English full spellings corresponding to the English abbreviation character strings.

15. The apparatus of claim 14, wherein the processor, when calculating the probability that the at least one candidate word is each an english spell of the english abbreviation string based on a mapping between the english abbreviation and the english spell, is configured to:

16. The apparatus of claim 15, wherein the english abbreviation prediction model employs a seq2seq model.

17. The apparatus of claim 15, wherein the processor, prior to entering the english abbreviation string into the english abbreviation prediction model, is further configured to:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

18. The apparatus of claim 17, wherein the processor, when labeling a correspondence between a sample english word and a sample english abbreviation in the sample data set, is configured to:

19. The apparatus of claim 18, wherein the processor, when calculating the probability that the at least one candidate word is abbreviated as the english abbreviation string based on a mapping between english abbreviations and english spellings, is configured to:

20. The apparatus of claim 14, wherein the processor, when determining at least one candidate word from the english word corpus that matches the greatest common factor sequence with the english abbreviated character string as the greatest common factor sequence, is configured to:

determining a target industry field where the field name is located;

21. The apparatus of claim 15, wherein the processor, when entering the english abbreviation string into an english abbreviation prediction model, is configured to:

determining a target industry field where the field name is located;

inputting an English abbreviation prediction model into the English abbreviation character string and the target industry field;

when calculating the probability that the at least one candidate word abbreviation is the English abbreviation character string based on the mapping relation between the English abbreviation and the English spelling, the method is used for:

22. The apparatus of claim 13, wherein the processor, in determining the abbreviated string of english contained in the field name, is configured to:

if the field name contains a separation character, dividing the field name into a plurality of character segments according to the separation character;

and determining a character segment which does not belong to the English word in the plurality of character segments as the English abbreviation character string.

23. The device of claim 13, wherein the processor is further configured to:

supplementing the field annotation corresponding to the field name to a database where the field name is located; or

24. The apparatus of claim 14, wherein the processor, prior to determining at least one candidate word from the english word corpus that matches the greatest common factor sequence with the english abbreviated character string as the greatest common factor sequence, is further configured to:

25. A character string understanding method, comprising:

acquiring a character string to be understood;

determining English abbreviation character strings contained in the character strings to be understood;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the character string to be understood based on the English full spelling corresponding to the English abbreviated character string to generate an understanding result of the character string to be understood.

26. The method of claim 25, wherein the string to be understood comprises: one or more of a field name in the database, a character string in the chat content, a professional term, and a search keyword.

27. A computing device comprising a memory and a processor;

the memory is to store one or more computer instructions;

the processor is coupled with the memory for executing the one or more computer instructions for:

acquiring a character string to be understood;

determining English abbreviation character strings contained in the character strings to be understood;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

28. A computer-readable storage medium storing computer instructions, which when executed by one or more processors, cause the one or more processors to perform the field annotation generation method of any one of claims 1-12 or the string understanding method of claim 25 or 26.

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, device, and storage medium for field annotation generation and string understanding.

Background

With the increasing amount of user data, more and more users have issued the requirement of data standardization, and it is expected that high-quality data assets are obtained. One of the key items of processing in the data normalization process is the completion of annotations to field names in the database.

At present, the field names in the database need to be annotated and completed in a manual mode, and the efficiency and the accuracy of the mode are low.

Disclosure of Invention

Aspects of the present application provide a field comment generation method, a character string understanding method, a device, and a storage medium, which are used to improve generation efficiency and accuracy of a field comment.

The embodiment of the application provides a field annotation generation method, which comprises the following steps:

acquiring a field name to be processed;

determining English abbreviation character strings contained in the field names;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the field name based on the English spelling corresponding to the English abbreviation character string to generate a field annotation of the field name.

The embodiment of the present application further provides a method for understanding a character string, including:

acquiring a character string to be understood;

determining English abbreviation character strings contained in the character strings to be understood;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

The embodiment of the application also provides a computing device, which comprises a memory and a processor;

the memory is to store one or more computer instructions;

the processor is coupled with the memory for executing the one or more computer instructions for:

acquiring a field name to be processed;

determining English abbreviation character strings contained in the field names;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the field name based on the English spelling corresponding to the English abbreviation character string to generate a field annotation of the field name.

The embodiment of the application also provides a computing device, which comprises a memory and a processor;

the memory is to store one or more computer instructions;

the processor is coupled with the memory for executing the one or more computer instructions for:

acquiring a character string to be understood;

determining English abbreviation character strings contained in the character strings to be understood;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the aforementioned field comment generation method or the aforementioned character string understanding method.

In the embodiment of the application, for the field names of the missing field comments, the english abbreviation character strings contained in the field names can be determined, and the english abbreviation character strings are restored to restore the english abbreviations in the field names to english spells, on the basis of which the field names can be translated in english to generate the field comments of the field names. Therefore, in the embodiment of the application, the field annotation does not need to be generated manually, the generation efficiency of the field annotation can be effectively improved, and the accuracy of the generated field annotation can be ensured by accurately restoring the Chinese and English abbreviations of the field names.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a flowchart illustrating a field annotation generation method according to an exemplary embodiment of the present application;

FIG. 2 is a logic block diagram of a field annotation generation method according to an exemplary embodiment of the present application;

fig. 3 is a schematic flowchart of a method for understanding a character string according to another exemplary embodiment of the present application;

FIG. 4 is a schematic block diagram of a computing device according to yet another exemplary embodiment of the present application;

fig. 5 is a schematic structural diagram of another computing device according to yet another exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

At present, the field names in the database need to be annotated and completed in a manual mode, and the efficiency and the accuracy of the mode are low. In view of these technical problems, the embodiments of the present application provide a solution, and one of the basic ideas is: for the field names with missing field comments, the English abbreviation character strings contained in the field names can be determined, and the English abbreviation character strings are restored to restore the English abbreviations in the field names to English spells, on the basis of which the field names can be translated in English to generate the field comments of the field names. Therefore, in the embodiment of the application, the field annotation does not need to be generated manually, the generation efficiency of the field annotation can be effectively improved, and the accuracy of the generated field annotation can be ensured by accurately restoring the Chinese and English abbreviations of the field names.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a flowchart illustrating a field annotation generation method according to an exemplary embodiment of the present application. Fig. 2 is a logic block diagram of a field annotation generation method according to an exemplary embodiment of the present application. The field comment generation method provided by the embodiment may be executed by a field comment generation apparatus, which may be implemented as software or as a combination of software and hardware, and may be integrally provided in a computing device. As shown in fig. 1 and 2, the method includes:

step 100, acquiring a field name to be processed;

step 101, determining English abbreviation character strings contained in field names;

102, determining English full spellings corresponding to English abbreviation character strings based on a mapping relation between the English abbreviations and the English full spellings;

and 103, performing English translation on the field name based on the English spelling corresponding to the English abbreviation character string to generate a field annotation of the field name.

The field comment generation method provided by this embodiment may be applied to other fields in a database, a spreadsheet, or the like, and the application scenario is not limited in this embodiment. Taking a database as an example, in most cases, columns in a data table may be referred to as fields, each of which contains information for a particular topic. Taking a spreadsheet as an example, a column in the spreadsheet may also be used as a field.

The field corresponds to a field name, which is the name of the information contained in the field. For example, in a database scenario, the field name may be the name of attribute class information, such as identification card, gender, and so on.

In practical applications, the field names are usually composed of characters in order to adapt to the requirements of software code technology and the like. Moreover, the writing manners of the field names may not be completely the same according to the habits of different technicians, which results in lower readthrough of the field names. Thus, field names are typically configured with field comments that explain the meaning of the field name. For example, the field name is yhsj, and the technician may add a field comment "user data" to the field name.

However, it appears that there are still a large number of field names for missing field annotations. The field names of the missing field annotations can only be understood manually by technicians, and particularly, the processing efficiency and the accuracy are low for the technicians which do not participate in the original development process.

In this embodiment, the field name of the missing field comment may be used as the field name to be processed. As mentioned above, in the present embodiment, the source of the field name to be processed is not limited.

In the present embodiment, the english abbreviation character string contained in the field name can be determined.

The english abbreviation character string may be a character string that cannot be translated into english words.

In practical applications, separation characters are usually present between the english abbreviation character strings belonging to different english words. For example, in the field name CUST _ NO, two english abbreviation character strings are separated by a separation character "_" s. In this case, the field name is considered to include two abbreviated character strings of english [ CUST ] and [ NO ].

In this embodiment, for each abbreviated english character string in the field name, the english spellings corresponding to the abbreviated english character string may be determined based on the mapping relationship between the abbreviated english character string and the english spellings.

The mapping relationship between the English abbreviation and the English full spelling in different industry fields may not be completely the same. In this embodiment, the english spellings corresponding to the english abbreviation character strings may be determined based on the mapping relationship between the english abbreviations and the english spellings in the target industry field according to the target industry field to which the field names belong.

Accordingly, based on the English spelling corresponding to the English abbreviated character string, the English translation can be carried out on the field name so as to generate the field annotation corresponding to the field name.

As mentioned above, the field name may contain an english abbreviation string, and may also contain other characters, such as an english spell string. In this embodiment, the abbreviated english character string in the field name may be replaced with a full english pinyin, and the full english pinyin corresponding to the abbreviated english character string may be combined with other characters in the field name, followed by performing english translation to generate a field annotation of the field name. Of course, the full English spelling corresponding to the abbreviated English character string can also be directly translated into the Chinese phrase, and the translation result is spliced with the understanding result of other characters in the field name to generate the field annotation of the field name.

And for the English words contained in the field names, English translation can be directly carried out without executing the operation of restoring the English abbreviation.

Accordingly, a field comment for the field name can be generated.

In this embodiment, for a field name with a missing field annotation, the english abbreviation string contained therein may be determined, and the english abbreviation string may be restored to restore the english abbreviation in the field name to an english spell. Therefore, in the embodiment of the application, the field annotation does not need to be generated manually, the generation efficiency of the field annotation can be effectively improved, and the accuracy of the generated field annotation can be ensured by accurately restoring the Chinese and English abbreviations of the field names.

In the above or below embodiments, at least one candidate word matching the greatest common factor sequence may be determined from the english abbreviation string as the greatest common factor sequence.

In practical application, English full spellings, namely English words, in different industry fields can be collected to form English word libraries in different industry fields.

Based on the english word libraries in different industry fields, in this embodiment, under the condition that the industry field to which the field name belongs is known, at least one candidate word matching the maximum common factor sequence can be determined from the english word library in the target industry field to which the field name belongs, with the english abbreviated character string as the maximum common factor sequence, and from the english word library corresponding to the target industry field.

Wherein, matching with the greatest common factor sequence means that all english letters contained in the english abbreviation exist in the english word, and although the english letters may not be continuous in the english word, the order of the english letters in the english word is identical to the order in the string of the english abbreviation.

For example, the english abbreviation string is cd, and for the english word code in the english word bank, there are english letters c and d, and the order of c and d in the code coincides with the order in the english abbreviation string cd, and therefore, the english word code can be determined as a candidate word for the english word string cd.

Therefore, a candidate word set corresponding to the English character string can be obtained, and the candidate word set comprises at least one candidate word.

For at least one candidate word in the candidate word set, the probability that the english abbreviation character string is used as the english abbreviation of the at least one candidate word may be calculated based on the mapping relationship between the english abbreviation and the english spell.

For this reason, in the present embodiment, the english abbreviation character string may be input into the english abbreviation prediction model, and the probability that the at least one candidate word is abbreviated as the english abbreviation character string may be calculated in the english abbreviation prediction model based on the mapping relationship between the english abbreviation and the english spell.

The english abbreviation prediction model may traverse each candidate word in the candidate word set and calculate a conditional probability of the candidate word under the english abbreviation string, that is, a probability of the candidate word being abbreviated as the english abbreviation string.

In the English abbreviation prediction model, the mapping relation between different English abbreviations and English spellings can be learned according to different industry fields. Here, the target industry field may be input into an english abbreviation prediction model in which a probability that at least one candidate word is abbreviated as an english abbreviation character string may be determined based on a mapping relationship between english abbreviations and english spellings in the target industry field to which the field names belong.

In order to enable the English abbreviation prediction model to learn the mapping relation between the English abbreviation and the English spelling in different industry fields, the English abbreviation prediction model can be trained. The training process may be:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

and inputting the labeled sample data set into an English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relation between English abbreviations and English spellings.

The method can be used for obtaining training texts from a network by adopting a crawler technology and the like, can also use an English abbreviation dictionary and an English word dictionary as the training texts, can classify the training texts according to the industry fields, and trains English abbreviation prediction models by using the training texts which are not identical aiming at different industry fields.

In the process of labeling the sample data set, the sample English abbreviation can be coded to obtain a coding sequence of the sample English abbreviation, wherein the coding sequence is used for representing common factors between the sample English abbreviation and a sample English word corresponding to the sample English abbreviation; and establishing a corresponding relation between the coding sequence and the sample English word so as to allow an English abbreviation prediction model to learn the mapping relation between the coding sequence and the English spelling.

In practical application, the sample english abbreviation can be encoded by 01, and the obtained encoding sequence will be 01 sequence. Wherein, 1 in the coding sequence can represent that there is a common factor between the sample English abbreviation and its corresponding sample English word at the sequence position. For example, for a sample english abbreviation cd and its corresponding sample english word code, the sample english abbreviation can be coded as [ 1010 ].

Therefore, the English abbreviation prediction model can learn the mapping relation between the coding sequence and the English full spelling from the labeled sample data in a centralized way.

Accordingly, the english abbreviation prediction model may calculate, as the probability that the at least one candidate word is abbreviated as the english abbreviation character string, based on the mapping relationship between the coding sequence and the english spell.

In practical application, which kind of coding sequence a candidate word may be abbreviated to may not be predicted, and therefore, a plurality of coding sequences that a candidate word may be abbreviated to may be determined exhaustively or according to a preset rule, and in an english abbreviation prediction model, probabilities that a candidate word is abbreviated to the aforementioned plurality of coding sequences are predicted respectively according to a mapping relationship between a learned coding sequence and an english spell. The plurality of coding sequences comprise coding sequences corresponding to English abbreviation character strings.

Of course, this is merely exemplary, and in this embodiment, it may also be specified that the english abbreviation prediction model calculates only the probability that the candidate word is abbreviated as the coding sequence corresponding to the english abbreviation character string, and does not calculate the probability that the candidate word is abbreviated as another coding sequence. This embodiment is not limited to this.

In the embodiment, the English abbreviation character string is represented in a coding sequence mode, and the prediction problem of the English abbreviation can be converted into a classification problem, so that the prediction efficiency is higher, and the prediction result is more reasonable and accurate.

In practical application, the english abbreviation prediction model may adopt seq2seq model. Of course, the english abbreviation prediction model may also adopt other types of algorithm models, and the embodiment is not limited thereto.

To this end, a probability that at least one candidate word in the candidate word set is each abbreviated as an english abbreviation character string may be determined based on an english abbreviation prediction model.

On the basis, the probability that the at least one candidate word is respectively used as the English spelling of the English abbreviated character string can be calculated according to the probability that the at least one candidate word is abbreviated as the English abbreviated character string and output by the English abbreviated prediction model based on the Bayesian hypothesis.

The calculation process based on the Bayesian assumption can be characterized as the following calculation formula:

where P (word | abbr) represents the probability that the word candidate word is an english spell of an english abbreviation character string abbr, P (abbr | word) represents the probability that the word candidate word is abbreviated as an english abbreviation character string abbr (from the aforementioned english abbreviation prediction model), P (word) represents the frequency of occurrence of the word candidate word, and P (abbr) represents the frequency of occurrence of the english abbreviation character string.

Accordingly, the probability that at least one candidate word in the candidate word set is respectively used as the English spelling of the English abbreviated character string can be calculated.

On the basis, the candidate words with the probability meeting the preset requirement can be used as English full spellings corresponding to the English abbreviated character strings. For example, the candidate word with the highest probability is used as the english spell corresponding to the english abbreviation string.

In the implementation, adaptive English word libraries can be constructed for different industry fields, so that the reduction process of the English abbreviations has a more accurate and reasonable reduction range, and the reduction accuracy of the English abbreviations can be effectively improved. Moreover, the English abbreviation character string is represented by adopting a coding sequence mode, so that the prediction problem of the English abbreviation can be converted into a classification problem, the prediction efficiency is higher, the prediction result is more reasonable and accurate, and the accuracy of reduction of the English abbreviation can be effectively improved.

In the above or below embodiments, the field name may contain a separator character.

In this embodiment, if the field name includes a separation character, the field name may be divided into a plurality of field segments according to the separation character; the character segment not belonging to the english word among the plurality of character segments may be determined as an english abbreviation character string.

The segmented characters in the field names serve in most cases as semantic segmentation. For example, the separator character "_" in the field name CUST _ NO functions as semantic division, dividing the semantics of the field name into clients and numbers.

In this embodiment, the field names may be divided according to the separator characters, and the understanding results may be generated for a plurality of the divided field. On the basis, the respective understanding results of the plurality of character segments can be spliced to generate the field annotation corresponding to the field name.

In addition, in the present embodiment, the separator characters in the field names may be retained in the field comments of the field names, or may be deleted directly and no longer appear in the field comments. This can be flexibly set according to actual requirements or user instructions, and the present embodiment does not limit this.

In the embodiment, the field names can be understood in a segmented manner, so that the field names can be understood more accurately, especially for the field names containing multiple semantics, the mutual influence among different semantics can be avoided, the multiple semantics contained in the field names are effectively ensured to obtain the most accurate understanding result, and the accuracy of the finally generated field annotations is effectively improved.

In the above or below embodiments, based on the field comments generated for the field names, the field comments corresponding to the field names may be supplemented into the database in which the field names are located.

Accordingly, the generated field annotation can be applied to the database, and the field annotation is added to the field name in the database.

In this embodiment, the association relationship between the field names and the field annotations in the database may also be constructed based on the field annotations corresponding to the field names and the field annotations corresponding to other field names in the database where the field names are located.

Based on the method, the association relationship between the field names and the field comments can be used as an intermediary in the process of accessing the database, so that the visitor can be ensured to correctly understand the meaning of each field name in the database.

In practical application, the association relationship between the field names and the field comments can be configured in a related data access protocol, so that the communication parties perform data processing according to the same understanding basis.

Of course, the application of the field annotation is by no means limited to this, and in the present embodiment, the generated field annotation may also be applied to other processing items, which are not exhaustive here.

In the above or following embodiments, in the process of determining an english spell corresponding to an english abbreviated character string based on a mapping relationship between an english abbreviation and an english spell, the english abbreviated character string may be further recognized by using an english abbreviation dictionary, and if it is determined that the english abbreviated character string exists in the english abbreviation dictionary, the english spell corresponding to the english abbreviated character string is determined according to the english abbreviation dictionary. Without performing operations such as determining candidate words.

The english abbreviation dictionary may be a dictionary commonly used in the industry field, and may also be a dictionary commonly used by other authorities or groups for authentication, which is not limited in this embodiment. The English abbreviation dictionary records the corresponding relation between English abbreviation and English spelling.

Accordingly, in this embodiment, it may be determined in advance whether the abbreviated english character string appears in the abbreviated english dictionary, if so, the full english pinyin may be determined directly according to the abbreviated english dictionary, and if not, the full english pinyin of the abbreviated english character string may be determined by determining at least one candidate word matching the maximum common factor sequence and other subsequent operations from the english word bank according to the abbreviated english character string provided in the foregoing embodiment as the maximum common factor sequence. This can effectively improve the efficiency of English abbreviation reduction.

Fig. 3 is a flowchart illustrating a method for understanding a character string according to another exemplary embodiment of the present application. The character string understanding method provided by the embodiment may be executed by a character string understanding apparatus, which may be implemented as software or as a combination of software and hardware, and may be integrally provided in a computing device. As shown in fig. 3, the method includes:

step 300, acquiring a character string to be understood;

step 301, determining English abbreviation character strings contained in the character strings to be understood;

step 302, determining English full spellings corresponding to the English abbreviation character strings based on the mapping relation between the English abbreviation and the English full spellings;

and step 303, performing English translation on the character string to be understood based on the English spelling corresponding to the English abbreviation character string to generate an understanding result of the character string to be understood.

The character string understanding method provided by the embodiment may be applied to a scene in which a character string with an unknown meaning is understood, for example, a database scene, a spreadsheet scene, a chat scene, a periodical translation or reading scene, a search engine scene, a shopping mall scene, and the like.

The type of string to be understood may not be exactly the same for different application scenarios. The string to be understood may include one or more of a field name in a database, a string in chat content, a specialized term, and a search keyword. For example, in a database scenario, the string to be understood may be a field name, and in a spreadsheet scenario, the string may be the content in any cell. In other scenarios, the character string to be understood may also be a string of characters in a text, or a sentence of code in a code file, etc.

Accordingly, in this embodiment, the character string to be understood may be any character string with unknown meaning, and the source, specification, type, and the like of the character string to be understood are not limited in this embodiment.

The character string understanding method provided by the embodiment can realize the reduction of the character string with unknown meaning. For example, in the IM tool, when a chat is typed, the abbreviated character string in the chat content is restored; academic journals or professional journals, such as hospital journals, for example, in which the abbreviations for the terms are reduced; and restoring the key word abbreviation in a search scene, such as commodity key words in a shopping mall scene or search key words in a search engine.

The present embodiment differs from the embodiment shown in fig. 1 in that the character string to be understood in the present embodiment is not limited to the field names in the foregoing embodiments.

Based on similar inventive concepts, the technical details in the present embodiment may refer to the related descriptions in the embodiments of the final understanding result generation method, and the detailed technical details will not be expanded for the sake of brevity, which should not cause a loss of the protection scope of the present application.

Only a few representative embodiments are described below by way of example.

In an alternative embodiment, the step of determining the english spellings corresponding to the english abbreviation character strings based on the mapping relationship between the english abbreviation and the english spellings includes:

calculating the probability that at least one candidate word is respectively used as the English full spelling of the English abbreviation character string based on the mapping relation between the English abbreviation and the English full spelling;

and taking the candidate words with the probability meeting the preset requirement as English full spellings corresponding to the English abbreviation character strings.

In an alternative embodiment, the step of calculating the probability that each of the at least one candidate word is an english spell of the string of english abbreviations based on the mapping relationship between the english abbreviations and the english spells includes:

inputting the English abbreviation character string into an English abbreviation prediction model; calculating the probability of at least one candidate word abbreviated as an English abbreviated character string in an English abbreviated prediction model based on the mapping relation between the English abbreviation and the English full spelling;

and calculating the probability that the at least one candidate word is respectively used as the English spelling of the English abbreviation character string according to the probability that the at least one candidate word is abbreviated as the English abbreviation character string and output by the English abbreviation prediction model based on Bayesian hypothesis.

In an alternative embodiment, the english abbreviation prediction model employs a seq2seq model.

In an alternative embodiment, before the step of inputting the english abbreviation character string into the english abbreviation prediction model, the method further comprises:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

In an alternative embodiment, the step of labeling the sample data set with a correspondence between a sample english word and a sample english abbreviation includes:

and establishing a corresponding relation between the coding sequence and the sample English word so as to allow an English abbreviation prediction model to learn the mapping relation between the coding sequence and the English spelling.

In an alternative embodiment, the step of calculating the probability that at least one candidate word abbreviation is an english abbreviation character string based on a mapping relationship between the english abbreviation and an english spell includes:

and calculating the probability of the coding sequence corresponding to the abbreviation of the at least one candidate word as the abbreviation of the English abbreviation character string based on the mapping relation between the coding sequence and the English full spelling.

In an alternative embodiment, the step of determining at least one candidate word matching the greatest common factor sequence from the english word library using the english abbreviation string as the greatest common factor sequence includes:

determining a target industry field where a character string to be processed is located;

and taking the English abbreviation character string as a maximum common factor sequence, and determining at least one candidate word matched with the maximum common factor sequence from an English word library corresponding to the target industry field.

In an alternative embodiment, the step of inputting the english abbreviation string into an english abbreviation prediction model comprises:

determining a target industry field where a character string to be processed is located;

english abbreviation character strings and an English abbreviation prediction model input by the target industry field;

calculating the probability of at least one candidate word abbreviation being an English abbreviation character string based on the mapping relationship between the English abbreviation and the English spelling, comprising:

and calculating the probability of the abbreviation of the at least one candidate word as the English abbreviation character string based on the mapping relation between the English abbreviation and the English full spelling in the target industry field.

In an alternative embodiment, the step of determining the english abbreviation string contained in the string to be processed includes:

if the character string to be processed contains the separation characters, dividing the character string to be processed into a plurality of character segments according to the separation characters;

and determining a character segment which does not belong to the English word in the plurality of character segments as an English abbreviation character string.

In an optional embodiment, the method further comprises:

supplementing the field annotation corresponding to the character string to be processed into a database in which the character string to be processed is located; or

And constructing an association relation between the character string to be processed and the field annotation in the database based on the field annotation corresponding to the character string to be processed and the field annotations corresponding to other character strings to be processed in the database where the character string to be processed is located.

and if the English abbreviated character string is determined not to exist in the English abbreviated dictionary, performing the operation of determining at least one candidate word matched with the maximum common factor sequence from the English word library by taking the English abbreviated character string as the maximum common factor sequence.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps 100 to 102 may be device a; for another example, the execution subject of steps 100 and 101 may be device a, and the execution subject of step 102 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 100, 102, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel.

Fig. 4 is a schematic structural diagram of a computing device according to another exemplary embodiment of the present application. As shown in fig. 4, the computing device includes: a memory 40 and a processor 41.

A processor 41, coupled to the memory 40, for executing the computer program in the memory 40 for:

acquiring a field name to be processed;

determining English abbreviation character strings contained in the field names;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the field name based on the English spelling corresponding to the English abbreviated character string to generate a field annotation of the field name.

In an alternative embodiment, the processor, when determining the english spellings corresponding to the english abbreviation character strings based on the mapping relationship between the english abbreviation and the english spellings, is configured to:

and taking the candidate words with the probability meeting the preset requirement as English full spellings corresponding to the English abbreviation character strings.

In an alternative embodiment, the processor, when calculating the probability that each of the at least one candidate word is an english spell of the string of english abbreviations based on the mapping relationship between the english abbreviations and the english spells, is configured to:

In an alternative embodiment, the english abbreviation prediction model employs a seq2seq model.

In an alternative embodiment, the processor is further configured to, prior to entering the english abbreviation string into the english abbreviation prediction model:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

In an alternative embodiment, the processor, when labeling the sample data set with a correspondence between a sample english word and a sample english abbreviation, is configured to:

In an alternative embodiment, the processor, when calculating the probability that the at least one candidate word is abbreviated as an english abbreviation string based on a mapping relationship between the english abbreviation and an english spell, is configured to:

In an alternative embodiment, the processor, when determining at least one candidate word matching the greatest common factor sequence from the english alphabet using the english abbreviation string as the greatest common factor sequence, is configured to:

determining a target industry field where the field name is located;

In an alternative embodiment, the processor, when entering the english abbreviation string into the english abbreviation prediction model, is configured to:

determining a target industry field where the field name is located;

english abbreviation character strings and an English abbreviation prediction model input by the target industry field;

calculating the probability that at least one candidate word is abbreviated as an English abbreviation character string based on the mapping relation between the English abbreviation and the English spelling, and the probability is used for:

In an alternative embodiment, the processor, in determining the abbreviated string in english contained in the field name, is configured to:

if the field name contains the separation character, dividing the field name into a plurality of character segments according to the separation character;

and determining a character segment which does not belong to the English word in the plurality of character segments as an English abbreviation character string.

In an alternative embodiment, the processor is further configured to:

supplementing the field comments corresponding to the field names to a database where the field names are located; or

And constructing the association relationship between the field names and the field annotations in the database based on the field annotations corresponding to the field names and the field annotations corresponding to other field names in the database where the field names are located.

In an alternative embodiment, the processor is further configured to, before determining at least one candidate word from the english word library that matches the greatest common factor sequence with the english abbreviation string as the greatest common factor sequence:

It should be noted that, for the above technical details in the embodiments of the computing device, reference may be made to the related description in the embodiments of the field annotation generation method, and for the sake of brevity, no further description is provided here, but this should not cause a loss of the scope of the present application.

Further, as shown in fig. 4, the computing device further includes: communication components 42, power components 43, and the like. Only some of the components are schematically shown in fig. 4, and the computing device is not meant to include only the components shown in fig. 4.

Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps that can be executed by a computing device in the foregoing method embodiments when executed.

Fig. 5 is a schematic structural diagram of another computing device according to yet another embodiment of the present application. As shown in fig. 5, the computing device includes: a memory 50 and a processor 51.

A processor 51, coupled to the memory 50, for executing the computer program in the memory 50 for:

acquiring a character string to be understood;

determining English abbreviation character strings contained in the character strings to be understood;

determining English full spellings corresponding to the English abbreviation character strings based on a mapping relation between the English abbreviation and the English full spellings;

and performing English translation on the character string to be understood based on the English spelling corresponding to the English abbreviation character string to generate an understanding result of the character string to be understood.

In an alternative embodiment, the processor 51, when determining the english spellings corresponding to the english abbreviation character strings based on the mapping relationship between the english abbreviation and the english spellings, is configured to:

and taking the candidate words with the probability meeting the preset requirement as English full spellings corresponding to the English abbreviation character strings.

In an alternative embodiment, the processor 51, when calculating the probability that each of the at least one candidate word is an english spell of the english abbreviation string based on the mapping relationship between the english abbreviation and the english spell, is configured to:

In an alternative embodiment, the english abbreviation prediction model employs a seq2seq model.

In an alternative embodiment, the processor 51 is further configured to, before entering the english abbreviation string into the english abbreviation prediction model:

acquiring a sample data set containing sample English words and sample English abbreviations;

labeling a corresponding relation between a sample English word and a sample English abbreviation in the sample data set;

In an alternative embodiment, the processor 51, when labeling the sample data set with a correspondence between a sample english word and a sample english abbreviation, is configured to:

In an alternative embodiment, the processor 51, when calculating the probability that at least one candidate word is abbreviated as an english abbreviation character string based on the mapping relationship between the english abbreviation and the english spell, is configured to:

In an alternative embodiment, the processor 51, when determining at least one candidate word matching the greatest common factor sequence from the english word library with the english abbreviation string as the greatest common factor sequence, is configured to:

determining a target industry field where a character string to be processed is located;

In an alternative embodiment, the processor 51, when entering the english abbreviation string into the english abbreviation prediction model, is operable to:

determining a target industry field where a character string to be processed is located;

english abbreviation character strings and an English abbreviation prediction model input by the target industry field;

In an alternative embodiment, the processor 51, when determining the english abbreviation string contained in the string to be processed, is configured to:

and determining a character segment which does not belong to the English word in the plurality of character segments as an English abbreviation character string.

In an optional embodiment, the method is further for:

supplementing the field annotation corresponding to the character string to be processed into a database in which the character string to be processed is located; or

In an alternative embodiment, the processor 51 is further configured to, before determining at least one candidate word matching the greatest common factor sequence from the english word library using the english abbreviation string as the greatest common factor sequence:

It should be noted that, for the sake of brevity, the above description of the technical details in the embodiments of the computing device may refer to the related descriptions in the embodiments of the string understanding method, which should not be repeated herein, but should not cause a loss of the scope of the present application.

Further, as shown in fig. 5, the computing device further includes: communication components 52, power components 53, and the like. Only some of the components are schematically shown in fig. 5, and the computing device is not meant to include only the components shown in fig. 5.

The memories of fig. 4 and 5 are used, among other things, to store computer programs and may be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, pictures, videos, and so forth. The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Wherein the communication components of fig. 4 and 5 are configured to facilitate wired or wireless communication between the device in which the communication components are located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

The power supply components of fig. 4 and 5, among other things, provide power to the various components of the device in which the power supply components are located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

22页详细技术资料下载

Method, device and storage medium for generating field annotation and understanding character string

相关技术

网友询问留言