阅读说明：本技术 字符识别方法及装置、电子设备和存储介质 (Character recognition method and device, electronic equipment and storage medium ) 是由 岳晓宇 旷章辉 蔺琛皓 孙红斌 张伟 于 2020-04-16 设计创作，主要内容包括：本公开涉及一种字符识别方法及装置、电子设备和存储介质,其中,所述方法包括：获取待识别的目标图像；基于确定的位置向量以及所述目标图像的第一图像特征,得到所述目标图像的字符特征；其中,所述位置向量是基于预设信息序列中字符的位置特征确定的；基于所述字符特征对所述目标图像中的字符进行识别,得到所述目标图像的字符识别结果。本公开实施例可以提高字符识别的准确率。(The present disclosure relates to a character recognition method and apparatus, an electronic device, and a storage medium, wherein the method includes: acquiring a target image to be identified; obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence; and identifying characters in the target image based on the character features to obtain a character identification result of the target image. The embodiment of the disclosure can improve the accuracy of character recognition.)

1. A character recognition method, comprising:

acquiring a target image to be identified;

obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence;

and identifying characters in the target image based on the character features to obtain a character identification result of the target image.

2. The method of claim 1, wherein obtaining the character feature of the target image based on the determined position vector and the first image feature of the target image comprises:

coding a first image characteristic of the target image to obtain a coding result of the first image characteristic;

determining a second image characteristic of the target image according to the encoding result of the first image characteristic;

and obtaining character features of the target image based on the determined position vector, the first image features and the second image features.

3. The method according to claim 2, wherein the encoding the first image feature of the target image to obtain the encoding result of the first image feature comprises:

and sequentially carrying out at least one stage of first coding processing on the plurality of first-dimension feature vectors of the first image features to obtain a coding result of the first image features.

4. The method according to claim 3, wherein the sequentially performing at least one stage of first encoding processing on the plurality of first-dimension feature vectors of the first image feature to obtain an encoding result of the first image feature comprises:

for one-level first coding processing in the at least one-level first coding processing, sequentially coding input information of the first coding nodes by using N first coding nodes to obtain output results of the N first coding nodes; under the condition that 1< i is not more than N, the input information of the ith first coding node comprises the output result of the (i-1) th first coding node, and N and i are positive integers;

and obtaining the coding result of the first image characteristic according to the output results of the N first coding nodes.

5. The method of claim 4, wherein the input information of the first coding node further comprises a first-dimension feature vector of the first image feature or an output result of a previous-stage first coding process.

6. The method according to any one of claims 2 to 5, wherein the obtaining character features of the target image based on the determined position vector, the first image features and the second image features comprises:

determining an attention weight according to the position vector and the second image feature;

and performing feature weighting on the first image features by using the attention weight to obtain the character features of the target image.

7. The method of any one of claims 1 to 6, further comprising:

acquiring a preset information sequence comprising at least one first preset information;

and sequentially carrying out at least one stage of second coding processing on the at least one piece of first preset information to obtain the position vector.

8. The method according to claim 7, wherein said sequentially performing at least one stage of second encoding processing on the at least one first preset information to obtain the position vector comprises:

for one-level second coding processing in the at least one-level second coding processing, sequentially coding input information of the second coding nodes by using M second coding nodes to obtain an output result of the Mth second coding node; under the condition that j is more than 1 and less than or equal to M, the input information of the jth second coding node comprises the output result of the (i-1) th second coding node, and M and j are positive integers;

and obtaining the position vector according to the output result of the Mth second coding node.

9. The method according to claim 8, wherein the input information of the second encoding node further includes the first preset information or an output result of a previous stage second encoding process.

10. The method according to any one of claims 1 to 9, wherein the recognizing the character in the target image based on the character feature to obtain a character recognition result of the target image comprises:

extracting semantic features of the target image;

and obtaining a character recognition result of the target image based on the semantic features and the character features of the target image.

11. The method of claim 10, wherein the extracting semantic features of the target image comprises:

on the basis of the acquired second preset information, sequentially determining semantic features of the target image at least one time step;

the obtaining of the character recognition result of the target image based on the semantic features and the character features of the target image includes:

and obtaining a character recognition result of the target image at least one time step based on the semantic features and the character features of the target image at least one time step.

12. The method according to claim 11, wherein the sequentially determining semantic features of the target image at least one time step based on the obtained second preset information comprises:

performing at least one stage of third coding processing on the second preset information to obtain semantic features of a first time step in the at least one time step;

and performing at least one stage of third coding processing on the character recognition result of the target image at the kth-1 time step to obtain the semantic features of the target image at the kth time step, wherein k is an integer greater than 1.

13. A character recognition apparatus, comprising:

the acquisition module is used for acquiring a target image to be identified;

the determining module is used for obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence;

and the recognition module is used for recognizing the characters in the target image based on the character features to obtain a character recognition result of the target image.

14. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any of claims 1 to 12.

15. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 12.

Technical Field

The present disclosure relates to the field of electronic technologies, and in particular, to a character recognition method and apparatus, an electronic device, and a storage medium.

Background

With the development of electronic technology, more and more work can be completed by electronic equipment or can be completed by assistance of the electronic equipment, which provides convenience for people. For example, the characters may be automatically recognized by a computer to improve the efficiency of manual processing.

Currently, character recognition may be performed on characters of a rule, for example, parsing a document, and the like. The character recognition can also be used for recognizing irregular characters, for example, irregular characters in natural scenes such as traffic signs, shop signboards and the like. However, due to factors such as viewing angle variation and illumination variation, it is difficult to accurately recognize irregular characters.

Disclosure of Invention

The present disclosure provides a character recognition technical solution.

According to an aspect of the present disclosure, there is provided a character recognition method including: acquiring a target image to be identified; obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence; and identifying characters in the target image based on the character features to obtain a character identification result of the target image.

In one possible implementation manner, the obtaining the character feature of the target image based on the determined position vector and the first image feature of the target image includes: coding a first image characteristic of the target image to obtain a coding result of the first image characteristic; determining a second image characteristic of the target image according to the encoding result of the first image characteristic; and obtaining character features of the target image based on the determined position vector, the first image features and the second image features. Here, the second image feature has a stronger position feature, so that the character feature of the obtained target image also has a stronger position feature, so that the character recognition result obtained by the character feature is more accurate, and the influence of the semantic meaning on the character recognition result is more reduced.

In a possible implementation manner, the encoding a first image feature of the target image to obtain an encoding result of the first image feature includes: and sequentially carrying out at least one stage of first coding processing on the plurality of first-dimension feature vectors of the first image features to obtain a coding result of the first image features. By sequentially carrying out one-stage or multi-stage first coding processing on a plurality of first-dimension feature vectors of the first image features, the position features included in the first image features can be enhanced, and the obtained coding result of the first image features can have more obvious position features among characters.

In a possible implementation manner, the sequentially performing at least one stage of first encoding processing on a plurality of first-dimension feature vectors of the first image feature to obtain an encoding result of the first image feature includes: for one-level first coding processing in the at least one-level first coding processing, sequentially coding input information of the first coding nodes by using N first coding nodes to obtain output results of the N first coding nodes; under the condition that 1< i is not more than N, the input information of the ith first coding node comprises the output result of the (i-1) th first coding node, and N and i are positive integers; and obtaining the coding result of the first image characteristic according to the output results of the N first coding nodes. Therefore, the input information of the first coding node can be transmitted to the last first coding node, so that the input information of the first coding node can be memorized for a long time, and the obtained output result is more accurate.

In one possible implementation manner, the input information of the first encoding node further includes a first-dimension feature vector of the first image feature or an output result of a previous-stage first encoding process. In this way, the first-level first coding processing can transmit the first-dimension feature vector of the first image feature or the output result of the previous-level first coding processing to the last first coding node through the first coding node, so that the output result of the first-level first coding processing can be more accurate.

In one possible implementation manner, the obtaining the character feature of the target image based on the determined position vector, the first image feature, and the second image feature includes: determining an attention weight according to the position vector and the second image feature; and performing feature weighting on the first image features by using the attention weight to obtain the character features of the target image. Here, the feature that needs to be focused in the first image feature may be enhanced in one step by the attention weight, so that the character feature obtained by performing feature weighting on the first image feature by using the attention weight may more accurately represent a more important feature portion in the first image feature.

In one possible implementation, the method further includes: acquiring a preset information sequence comprising at least one first preset information; and sequentially carrying out at least one stage of second coding processing on the at least one piece of first preset information to obtain the position vector. Since the at least one first preset information is sequentially encoded in the second encoding process of the at least one first preset information by using the neural network, the generated position vector is related to the order of the at least one first preset information, so that the position vector can represent the position characteristics between the characters.

In a possible implementation manner, the sequentially performing at least one level of second encoding processing on the at least one piece of first preset information to obtain the position vector includes: for one-level second coding processing in the at least one-level second coding processing, sequentially coding input information of the second coding nodes by using M second coding nodes to obtain an output result of the Mth second coding node; under the condition that j is more than 1 and less than or equal to M, the input information of the jth second coding node comprises the output result of the (i-1) th second coding node, and M and j are positive integers; and obtaining the position vector according to the output result of the Mth second coding node. Therefore, the input information of the first second coding node can be transmitted to the last second coding node, so that the input information of the second coding node is memorized for a long time, and the position vector is obtained more accurately.

In a possible implementation manner, the input information of the second encoding node further includes the first preset information or an output result of a previous-stage second encoding process. Therefore, the first-stage second coding processing can transmit the first preset information or the output result of the previous-stage second coding processing to the last first coding node through the second coding node, so that the output result of the first-stage first coding processing can be more accurate.

In a possible implementation manner, the recognizing the character in the target image based on the character feature to obtain a character recognition result of the target image includes: extracting semantic features of the target image; and obtaining a character recognition result of the target image based on the semantic features and the character features of the target image. Therefore, in the process of obtaining the character recognition result of the target image, the semantic features and the character features can be combined to provide the accuracy of the character recognition result.

In one possible implementation, the extracting semantic features of the target image includes: on the basis of the acquired second preset information, sequentially determining semantic features of the target image at least one time step; the obtaining of the character recognition result of the target image based on the semantic features and the character features of the target image includes: and obtaining a character recognition result of the target image at least one time step based on the semantic features and the character features of the target image at least one time step. Here, in the case where there are a plurality of characters in the target image, the character recognition result can be obtained in order of the position (character feature) and the semantic meaning (semantic feature) of the character, so that the accuracy of the character recognition result can be improved.

In a possible implementation manner, the sequentially determining semantic features of the target image at least one time step based on the obtained second preset information includes: performing at least one stage of third coding processing on the second preset information to obtain semantic features of a first time step in the at least one time step; and performing at least one stage of third coding processing on the character recognition result of the target image at the kth-1 time step to obtain the semantic features of the target image at the kth time step, wherein k is an integer greater than 1. By the mode, the input information of the third coding node which is sequenced at the front can be transmitted to the third coding node which is sequenced at the back, so that the input information of the third coding node can be memorized for a long time, and the obtained semantic features are more accurate.

According to an aspect of the present disclosure, there is provided a character recognition apparatus including:

the acquisition module is used for acquiring a target image to be identified;

the determining module is used for obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence;

and the recognition module is used for recognizing the characters in the target image based on the character features to obtain a character recognition result of the target image.

In a possible implementation manner, the determining module is specifically configured to encode a first image feature of the target image to obtain an encoding result of the first image feature; determining a second image characteristic of the target image according to the encoding result of the first image characteristic; and obtaining character features of the target image based on the determined position vector, the first image features and the second image features.

In a possible implementation manner, the determining module is specifically configured to perform at least one stage of first encoding processing on the plurality of first-dimension feature vectors of the first image feature in sequence to obtain an encoding result of the first image feature.

In a possible implementation manner, the determining module is specifically configured to, for a first-level first encoding process in the at least one-level first encoding process, sequentially encode input information of the first encoding node by using N first encoding nodes, and obtain output results of the N first encoding nodes; under the condition that 1< i is not more than N, the input information of the ith first coding node comprises the output result of the (i-1) th first coding node, and N and i are positive integers; and obtaining the coding result of the first image characteristic according to the output results of the N first coding nodes.

In one possible implementation manner, the input information of the first encoding node further includes a first-dimension feature vector of the first image feature or an output result of a previous-stage first encoding process.

In a possible implementation manner, the determining module is specifically configured to determine an attention weight according to the position vector and the second image feature; and performing feature weighting on the first image features by using the attention weight to obtain the character features of the target image.

In one possible implementation, the apparatus further includes: the encoding module is used for acquiring a preset information sequence comprising at least one piece of first preset information; and sequentially carrying out at least one stage of second coding processing on the at least one piece of first preset information to obtain the position vector.

In a possible implementation manner, the encoding module is specifically configured to, for a first-level second encoding process in the at least one-level second encoding process, sequentially encode input information of the second encoding node by using M second encoding nodes, and obtain an output result of an M-th second encoding node; under the condition that j is more than 1 and less than or equal to M, the input information of the jth second coding node comprises the output result of the (i-1) th second coding node, and M and j are positive integers; and obtaining the position vector according to the output result of the Mth second coding node.

In a possible implementation manner, the input information of the second encoding node further includes the first preset information or an output result of a previous-stage second encoding process.

In a possible implementation manner, the recognition module is specifically configured to extract semantic features of the target image; and obtaining a character recognition result of the target image based on the semantic features and the character features of the target image.

In a possible implementation manner, the identification module is specifically configured to sequentially determine semantic features of the target image at least one time step based on the obtained second preset information; and obtaining a character recognition result of the target image at least one time step based on the semantic features and the character features of the target image at least one time step.

In a possible implementation manner, the identification module is specifically configured to perform at least one stage of third encoding processing on the second preset information to obtain a semantic feature of a first time step in the at least one time step; and performing at least one stage of third coding processing on the character recognition result of the target image at the kth-1 time step to obtain the semantic features of the target image at the kth time step, wherein k is an integer greater than 1.

According to an aspect of the present disclosure, there is provided an electronic device including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the above character recognition method is performed.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above character recognition method.

In the embodiment of the disclosure, a target image to be recognized may be obtained, then character features of the target image are obtained based on the determined position vector and the first image features of the target image, and then characters in the target image are recognized based on the character features, so as to obtain a character recognition result of the target image. The position vector is determined based on the position features of the characters in the preset information sequence and can represent the position features among the characters, so that the influence of the position features among the characters on character recognition results can be increased in the character recognition process, the character recognition accuracy is improved, and for example, a good recognition effect can be obtained for irregular characters and non-semantic characters.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a flow chart of a character recognition method according to an embodiment of the present disclosure.

Fig. 2 illustrates a block diagram of an example of determining a second image feature of a target image according to an embodiment of the present disclosure.

Fig. 3 shows a block diagram of an example of obtaining character recognition results using a neural network according to an embodiment of the present disclosure.

Fig. 4 shows a block diagram of an example of a character recognition apparatus according to an embodiment of the present disclosure.

Fig. 5 shows a block diagram of an example of a character recognition apparatus according to an embodiment of the present disclosure.

Fig. 6 shows a block diagram of an example of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

According to the character recognition scheme provided by the embodiment of the disclosure, the target image to be recognized can be obtained, then the character features of the target image are obtained based on the determined position vector and the first image features of the target image, and then the characters in the target image are recognized based on the character features, so that the character recognition result of the target image is obtained. The position vector is determined based on the position characteristics of the characters in the preset information sequence and can be used for representing the position characteristics of the characters, so that the position characteristics among the characters can be enhanced in the character recognition process, and the obtained character recognition result is more accurate.

In the related art, character sequences are generally recognized through semantic features between characters, but some characters in the character sequences are less semantically related, for example, characters in character sequences such as license plate numbers, room numbers and the like are less semantically related, so that the character sequences are recognized through the semantic features less effectively. The character recognition scheme provided by the embodiment of the disclosure can enhance the influence of the position characteristics of the characters on the character recognition, reduce the dependence of the character recognition process on the semantic characteristics, and has a better recognition effect on the recognition of characters with less semantic association or the recognition of irregular characters.

The technical scheme provided by the embodiment of the disclosure can be applied to the expansion of application scenes such as character recognition in an image, image-text conversion and the like, and the embodiment of the disclosure does not limit the application scenes. For example, irregular characters in the traffic sign are subjected to character recognition to determine the traffic indication represented by the traffic sign, so that convenience is provided for users.

Fig. 1 shows a flow chart of a character recognition method according to an embodiment of the present disclosure. The character recognition method may be performed by a terminal device, a server, or other types of electronic devices, where the terminal device may be a User Equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. In some possible implementations, the character recognition method may be implemented by a processor calling computer readable instructions stored in a memory. The following describes a character recognition method according to an embodiment of the present disclosure, taking an electronic device as an execution subject.

In step S11, a target image to be recognized is acquired.

In the embodiment of the present disclosure, the electronic device may have an image capturing function, and may capture a target image to be recognized. Alternatively, the electronic device may acquire the target image to be recognized from another device, for example, the electronic device may acquire the target image to be recognized from an image pickup device, a monitoring device, or the like. The target image to be recognized may be an image waiting for character recognition. The target image may carry characters, and the characters may be single characters or character strings. The characters in the target image may be regular characters, for example, text written in a canonical font may be regular characters. Regular characters can have the characteristics of being arranged orderly, uniform in size, not deformed, not shielded and the like. In some implementations, the characters in the target image may also be irregular characters, such as artistic characters on shop signboards and advertising covers. Irregular characters may have features such as being out of order, not in size, deformed or occluded.

Step S12, obtaining character features of the target image based on the determined position vector and the first image features of the target image; wherein the position vector is used for representing the position characteristics of the character.

In the embodiment of the present disclosure, a position vector for representing a position feature of a character may be determined based on the position feature of the character in a preset information sequence, for example, a preset information sequence with a certain length may be obtained, and then the position feature of the character in the preset information sequence may be extracted. The position vector is related to the position of the character, for example, the position of a character to be recognized in the character sequence is the third character position, and the position vector may indicate the relative position of the character to be recognized in the character sequence, that is, the third character position. To reduce the relevance of the position vector to the character semantics, the characters in the preset information sequence may be identical. In some implementations, each character in the preset information sequence can also be set to information without semantics, thereby further reducing the correlation of the position vector with the character semantics. The position vectors are less semantically related to the characters, so that the position vectors may be the same or different for different target images.

The first image feature of the target image may be obtained by performing image extraction on the target image, for example, the first image feature of the target image may be obtained by performing at least one convolution operation on the target image by using a neural network. According to the determined position vector and the first image feature of the target image, the character feature of the target image can be determined, for example, the determined position vector and the first image feature of the target image are fused to obtain the character feature of the target image. Here, since the character feature is obtained based on the position vector and the first image feature, the character feature is less affected by the semantics of the character.

Step S13, recognizing the characters in the target image based on the character features, and obtaining a character recognition result of the target image.

In the embodiment of the present disclosure, the neural network may be used to process the character features, for example, to perform an activation operation on the character features, or to perform a full connection operation on a full connection layer of the neural network to output the character features, so as to obtain a character recognition result of the target image. The character recognition result may be a recognition result for a character in the target image. In the case where one character is included in the target image, the character recognition result may be one character. In the case where a character sequence is included in the target image, the character recognition result may be one character sequence in which the order of each character is the same as the order of the corresponding character in the target image.

The character recognition result obtained through the character features is less influenced by the semantics of the characters, so that a better recognition effect can be achieved for character sequences with less semantic association among some characters, for example, character recognition can be performed for character sequences without semantic association in a license plate.

In step S12, the character feature of the target image may be obtained based on the determined position vector and the first image feature of the target image, so as to reduce the influence of the semantic meaning on the character feature. The following provides one implementation of obtaining character features of a target image.

In one possible implementation manner, a first image feature of a target image may be encoded to obtain an encoding result of the first image feature, a second image feature of the target image is determined according to the encoding result of the first image feature, and then a character feature of the target image is obtained based on a preset position vector, the first image feature, and the second image feature.

In this implementation, the first image feature of the target image may be encoded using a neural network, for example, the first image feature may be encoded row by row or column by column, so that the position feature included in the first image feature may be enhanced. Then, according to the encoding result obtained by encoding the first image feature, a second image feature of the target image may be obtained, for example, the first image feature and the encoding result may be fused to obtain the second image feature of the target image, where the second image feature has a stronger position feature than the first image feature. And then, the character features of the target image can be obtained based on the determined position vector, the first image features and the second image features, for example, the determined position vector, the first image features and the second image features are fused to obtain the character features of the target image, and the second image features have stronger position features, so that the obtained character features of the target image also have stronger position features, the character recognition result obtained by the character features is more accurate, and the influence of semantics on the character recognition result is further reduced.

In the foregoing implementation, the first image feature of the target image may be encoded, so that the position feature included in the first image feature is enhanced, and a process of obtaining an encoding result of the first image feature is described below by using an example.

In one example, at least one level of first encoding processing may be sequentially performed on a plurality of first-dimension feature vectors of the first image feature, so as to obtain an encoding result of the first image feature.

In this example, the first image feature may include a plurality of first-dimension feature vectors. The first image feature may include features in multiple dimensions, e.g., the first image feature may include multiple dimensions of length, width, depth, etc. The feature dimensions may differ in different dimensions. The first-dimension feature vector may be a feature of the first image feature in one dimension, for example, the first-dimension feature vector may be a feature in a length dimension or a width dimension. The first encoding process may be encoding for a first image feature, and accordingly, the neural network may include at least one first encoding layer, and the encoding process corresponding to the first encoding layer may be the first encoding process. Here, the neural network may sequentially perform one-stage or multi-stage first encoding processing on the plurality of first-dimension feature vectors to obtain processing results of the plurality of first-dimension feature vectors, one first-dimension feature vector may correspond to one processing result, and then the plurality of processing results of the plurality of first-dimension features may be combined to form an encoding result of the first image feature. By sequentially carrying out one-stage or multi-stage first coding processing on a plurality of first-dimension feature vectors of the first image features, the position features included in the first image features can be enhanced, and the obtained coding result of the first image features can have more obvious position features among characters.

In this example, for a first encoding process of at least one first encoding process, the input information of the first encoding node may be sequentially encoded by using N first encoding nodes, so as to obtain output results of the N first encoding nodes; and under the condition that 1< i is less than or equal to N, the input information of the ith first coding node comprises the output result of the (i-1) th first coding node, and N and i are positive integers. And obtaining the coding result of the first image characteristic according to the output results of the N first coding nodes.

In this example, the neural network may be used to perform at least one level of first encoding processing on the first image feature to obtain an encoding result of the first image feature. The neural network may include at least one stage of a first coding layer, and the first coding layer may perform a first coding process, each stage of the first coding process being implemented by a plurality of coding nodes. In the case where the first encoding process is multi-stage, the operation performed by the first encoding process of each stage may be the same. For a level of first encoding processing in at least one level of first encoding processing, N first encoding nodes may be used to sequentially encode input information of the level of first encoding processing, one first encoding node may correspond to one input information, and input information of different first encoding nodes may be different. Accordingly, a first coding node may obtain an output result. The input information of the first encoding node in the first-stage first encoding process may be a first-dimension feature vector of the first image feature. The output result of the first coding node in the first-stage first coding processing can be used as the input information of the first coding node with the same sequence in the second-stage first coding processing, and so on until the last-stage first coding processing. The output result of the first coding node in the last-stage first coding process may be a processing result of the first-dimension feature vector described above. The first encoding process of one level may include N first encoding nodes, and in a case that 1< i ≦ N, that is, in a case that the first encoding node is a first encoding node other than the first encoding node in the first encoding process of the current level, the input information of the first encoding node may further include an output result of a previous first encoding node in the first encoding process of the level, so that the input information of the first encoding node may be transferred to the last first encoding node, thereby enabling the input information of the first encoding node to be memorized for a long time, and enabling the obtained output result to be more accurate.

Fig. 2 illustrates a block diagram of an example of determining a second image feature of a target image according to an embodiment of the present disclosure. In this example, a neural network, such as a Long Short-Term Memory network (LSTM), may be utilized to encode the first image feature F of the target image. The neural network may include two first coding layers, each of which may include a plurality of first coding nodes (corresponding to the coding nodes in fig. 2). Here, the first image feature F of the target image may be input to a first coding layer of the neural network, and a plurality of first-dimension feature vectors (width-dimension feature vectors) of the first image feature F may be encoded by a plurality of first coding nodes of the first coding layer, respectively, to obtain an output result of each first coding node. The input information of the first coding node is the first dimension feature vector, the input information of the second first coding node is the output result of the first coding node and the second first dimension feature vector, and so on, the output result of the last first coding node can be obtained. The output results of the plurality of first coding points are input into the second layer first coding layer, and the processing procedure of the second layer first coding layer is similar to that of the first layer first coding layer, and is not repeated here. Finally, the coding result F of the first image characteristic can be obtained². The first image feature F and the result F of the encoding of the first image feature may then be combined²Performing feature fusion, wherein the feature fusion can be feature addition or combination to obtain a second image feature of the target image

Here, taking the example of two-layer LSTM encoding a first image feature F of a target image, a second image feature is obtained from the first image feature FCan be determined by the following equation:

wherein f is_i,jMay be a feature vector (first-dimension feature vector) of the first image feature F at the (i, j) position;can represent the output result F of the first layer and the first coding layer¹A feature vector at the (i, j) location;can represent the output result F¹A feature vector at the (i, j-1) position;can represent the encoding result F²A feature vector at the (i, j) location;can represent the encoding result F²A feature vector at the (i, j) location;the resulting second image feature may be represented;an addition operation of the vectors can be represented. Here, i and j are both natural numbers.

In the above implementation, the character feature of the target image may be obtained based on the determined position vector, the first image feature and the second image feature, and an example is provided below to explain a process of obtaining the character feature of the target image.

In one example, the attention weight may be determined according to the determined position vector and the second image feature, and then the first image feature may be feature-weighted by the attention weight, so as to obtain the character feature of the target image.

In one example, since the location vector and the second image feature both include a salient location feature, an attention weight may be determined from the location vector and the second image feature, e.g., a correlation of the location vector with the second image feature is determined, from which the attention weight is determined. The correlation of the position vector with the second image feature may be obtained by a dot multiplication of the position vector with the second image feature. The first image feature may be feature weighted using the determined attention weight, for example, the attention weight may be multiplied by the first image feature and summed to obtain the character feature of the target image. The attention weight can be used for enhancing the feature needing attention in the first image feature in one step, so that the character feature obtained by performing feature weighting on the first image feature by using the attention weight can more accurately represent the more important feature part in the first image feature.

In this example, the attention weight may be determined by the following equation (4):

wherein the content of the first and second substances,representing an attention weight; softmax stands forActivating a function; h is_t ^TRepresents a position vector h_tTransposing;representing a second image featureA feature vector at feature location (i, j). Using equation (2) above, the attention weight can be determined from the position vector and the second image feature.

In this example, the character characteristics can be determined by the following equation (5):

wherein, g_tRepresenting character features;representing an attention weight; f. of_i,jA feature vector representing the first image feature F at feature position (i, j). With the above formula (5), the character feature can be obtained from the attention weight and the first image feature.

In the above implementation, the attention weight may be determined based on the determined position vector and the second image feature. The position vector may represent a positional characteristic of the characters, i.e., may represent a relative position between the characters. The process of determining a position vector is described below by way of one implementation.

In a possible implementation manner, a preset information sequence including at least one first preset information may be obtained, and then at least one level of second encoding processing is performed on the at least one first preset information in sequence, so as to obtain a position vector.

In this implementation, the preset information sequence may include one or more first preset information. The first preset information may be information set according to an actual scene, and may not have a specific meaning. For example, the first preset information may be a counting instruction. The neural network can be used for sequentially carrying out one-stage or multi-stage second coding processing on at least one first preset message to obtain a position vector. Because the at least one piece of first preset information is the same and has no specific meaning, the semantic correlation between the at least one piece of first preset information is small, and the position vector obtained by sequentially carrying out one-stage or multi-stage second coding processing on the at least one piece of first preset information has low semantic correlation. Meanwhile, since the at least one first preset information is sequentially encoded in the process of performing the second encoding process on the at least one first preset information by using the neural network, the generated position vector is related to the order of the at least one first preset information, that is, it can be understood that the position vector is related to the position between the at least one first preset information, so that the position vector can represent the position characteristics between characters.

In an example of this implementation manner, for a level of second encoding processing in at least one level of second encoding processing, M second encoding nodes may be used to sequentially encode input information of the second encoding nodes, so as to obtain an output result of the M second encoding node. And under the condition that j is more than or equal to 1 and less than or equal to M, the input information of the jth second coding node comprises the output result of the (i-1) th second coding node, and M and j are positive integers. And obtaining a position vector according to the output result of the Mth second coding node.

In this example, the neural network may be used to sequentially perform one or more stages of second encoding processes on the at least one first preset information to obtain the position vector. In the case where the second encoding process is multi-stage, the operation performed by the second encoding process of each stage may be the same. For a level of second encoding processing in at least one level of second encoding processing, M second encoding nodes may be used to sequentially encode input information of the level of second encoding processing, one second encoding node may correspond to one input information, and input information of different second encoding nodes may be different. Accordingly, a second encoding node may obtain an output result. The input information of one of the second coding nodes in the first-level second coding process may be a first preset information. The output result of the second coding node in the first-stage first coding process can be used as the input information of the second coding node with the same sequence in the second-stage second coding process, and so on until the last-stage second coding process. The output result of the last second coding node in the last second coding process may be used as the position vector, or the output result of the last second coding node in the last second coding process may be further processed by convolution, pooling, and the like, so as to obtain the position vector. The first-stage second coding process may include M second coding nodes, and in a case that 1< j ≦ M, that is, in a case that the second coding node is another second coding node except for the first second coding node in the current-stage second coding process, the input information of the second coding node may further include an output result of the previous second coding node in the second coding process, so that the input information of the first second coding node may be transferred to the last second coding node, so that the input information of the second coding node is long-term memorized, and the obtained position vector is more accurate.

Here, the first preset information is taken as a constant "<next>", the case where the second encoding process is the second-order LSTM, for example, the position vector h can be determined using the following formula (6) and formula (7)_t。

h′_t＝LSTM(<next>,h′_t-1) Formula (6);

h_t＝LSTM(h′_t,h_t-1) Formula (7);

wherein, h'_tCan represent the output result of the t second coding node in the first-level second coding processing; h'_t-1The output result of the t-1 second coding node in the first-level second coding processing is shown; h is_tMay represent the output result of the tth second coding node in the second-level second coding process, i.e., a position vector; h is_t-1And the output result of the t-1 second coding node in the second-level second coding processing is shown. Wherein t is a natural number.

It should be noted that the process of obtaining the position vector from the at least one first preset information may be implemented by using the neural network shown in fig. 2, where the position vector may be an output result of the last second coding node in the second-stage second coding process, instead of being formed by output results of a plurality of second coding nodes together.

In step S103, the characters in the target image may be recognized based on the character features, and a character recognition result of the target image is obtained. In order to improve the accuracy of the character recognition result, the semantic features of the characters in the target image can be considered in the process of recognizing the characters in the target image. The following describes a process of obtaining a character recognition result of a target image by one implementation.

In one possible implementation, the semantic features of the target image may be extracted, and then the character recognition result of the target image is obtained based on the semantic features and the character features of the target image.

In this implementation manner, semantic features of the target image may be extracted, for example, semantic features of the target image may be extracted by using semantic extraction models of some scenes, and then the semantic features and the text features of the target image are fused to obtain a fusion result, for example, the semantic features and the text features may be spliced, or after the semantic features and the text features are spliced, feature weighting is performed to obtain a fusion result. Here, the weight of the feature weighting may be preset, or may be calculated from semantic features and character features. Then, a character recognition result of the target image can be obtained according to the fusion result, for example, a convolution operation, a full join operation, or the like can be performed on the fusion result at least once, so that a character recognition result of the target image can be obtained. Therefore, in the process of obtaining the character recognition result of the target image, the semantic features and the character features can be combined to provide the accuracy of the character recognition result.

For example, a semantic feature may be represented as c_tThe character feature can be expressed as g_tThe following formula (8) and formula (9) can be used to obtain the fusion result of semantic features and text features:

w_t＝softmax(W_f[g_t；c_t]+b_f) Formula (9);

wherein the content of the first and second substances,may represent the fusion result; w is a_tMay represent a semantic feature c_tAnd character characteristics g_tPerforming feature weighted weights; w_fMay represent a first mapping matrix, which may be a semantic feature c_tAnd character characteristics g_tMapping to a two-dimensional vector space; b_fA first bias term may be represented.

In obtaining a fusion resultThen, the following formula (10) can be used to obtain the character recognition result of the target image:

wherein, y_tThe character recognition result can be represented; w may represent a second mapping matrix, which may be applied to the fused resultsPerforming linear transformation; b may be a second bias term.

In an example of this implementation, the semantic features of the target image at least one time step may be sequentially determined based on the obtained second preset information, and then the character recognition result of the target image at least one time step may be obtained based on the semantic features and the character features of the target image at least one time step.

In this example, the obtained second preset information may be selected according to an actual scene, and the second preset information may not have a specific meaning. For example, the second preset information may be a start instruction. The step length of the time step can be set according to the actual application requirement. One semantic feature can be determined every time step, and the semantic features obtained at different time steps can be different. Here, the neural network may be used to encode the second preset information, to sequentially obtain semantic features of at least one time step, and then, according to the semantic features of the target image at the at least one time step and the character features of the at least one time step, a character recognition result of the target image at the at least one time step may be obtained. The semantic features of one time step and the character features of the same time step may correspond to the character recognition result of one time step, that is, in the case that there are a plurality of characters in the target image, the character recognition result may be sequentially obtained according to the positions (character features) and semantics (semantic features) of the characters, so that the accuracy of the character recognition result may be improved.

In this example, at least one level of third encoding processing may be performed on the second preset information to obtain a semantic feature of a first time step in at least one time step, and then at least one level of third encoding processing may be performed on a character recognition result of the target image at a k-1 time step to obtain a semantic feature of the target image at the k time step. Wherein k is an integer greater than 1.

In this example, the second preset information may be used as input information of at least one stage of the third encoding process in the neural network. Each level of the third encoding process may include a plurality of third encoding nodes, and each of the third encoding nodes may correspond to input information at one time step. The input information for different third coding nodes may be different. Accordingly, a third encoding node may obtain an output result. The input information of the first third coding node in the first-stage third coding process may be second preset information. The output result of the third coding node in the first-stage third coding processing may be used as the input information of the third coding node in the same order in the second-stage third coding processing, and so on until the last-stage third coding processing, so that at least one stage of third coding processing may be performed on the second preset information to obtain the output result of the first third coding node in the last-stage third coding processing, and the output result may be a semantic feature of the first time step in at least one time step. And further obtaining a character recognition result of the first time step according to the semantic features of the first time step and the character features of the same time step. The input information of the second third coding node in the first-stage third processing may be a character recognition result of the first time step. Then, at least one stage of third coding processing can be carried out on the character recognition result of the first time step to obtain the semantic features of the second time step. And further obtaining a character recognition result of the second time step according to the semantic features of the second time step and the character features of the same time step. And so on until the third encoding process of the last level. In the third encoding process of the last stage, the output result of the last third encoding node may be a semantic feature of the last time step. Namely, at least one level of third coding processing is carried out on the character recognition result of the target image at the k-1 time step, so that the semantic features of the target image at the k time step can be obtained. In the case that k is an integer greater than 1, that is, in the case that the third coding node is another third coding node except the first third coding node in the third coding process of the current stage, the input information of the third coding node may further include the output result of the previous third coding node in the third coding process of the stage, so that the input information of the third coding node ordered before may be transferred to the third coding node ordered after, and thus the input information of the third coding node may be memorized for a long time, so that the obtained semantic features are more accurate.

It should be noted that the process of determining the semantic feature from the second preset information may be implemented by using a neural network shown in fig. 2, where the semantic feature at the kth time step may be an output result of the kth third coding node in the second-stage third coding process.

In the embodiment of the disclosure, the neural network can be used to obtain the character recognition result of the target image. The following describes a process of obtaining a character recognition result of a target image by using a neural network by way of an example.

Fig. 3 shows a block diagram of an example of obtaining character recognition results using a neural network according to an embodiment of the present disclosure. In this example, the neural network may include an encoder and a decoder. First, the target image may be output to an encoder of the neural network, and the encoder may be used to extract an image feature of the target image, so as to obtain a first image feature F of the target image. Here, the image feature extraction may be performed on the target image by using a Network architecture of a 31-layer Residual Neural Network (ResNet). The encoder may include a position information enhancement module, and the position information enhancement module may be used to enhance the position information in the first image feature to obtain a second image feature of the target imageThe network architecture of the location information augmentation module may be as shown in fig. 2. The second image may then be characterizedInputting the attention module of the decoder, and characterizing the second image by the attention moduleAnd a position vector h_tAnd performing matrix multiplication and activation operation to obtain an attention weight, and then performing feature weighting on the first image feature F by using the attention weight, namely performing matrix multiplication on the attention weight and the first image feature to obtain the character feature of the target image. The decoder also comprises a dynamic fusion module, the character features and the semantic features can be fused by the dynamic fusion module, and then the fusion result is input into the full-link layer, so that the character recognition result can be obtained.

Here, the decoder also includes a position coding module, which can encode a plurality of constants "<next>"(first preset information) is input to the position coding module in turn, i.e. a constant is input at each time step"<next>". The position-coding module may comprise two coding layers (corresponding to the first coding process), which may be input "<next>"encode to get the position of the t time stepVector h_t. Here, the position encoding module may include two encoding layers. The decoder also comprises a semantic module which can convert a special token "<start>"(second preset information) is input as input information of a first time step and is input into the semantic module, and semantic features of the first time step output by the semantic module are obtained. Then the character recognition result y of the first time step₀The semantic feature of the second time step output by the semantic module can be obtained as the output result of the second time step of the semantic module, and by analogy, the semantic feature c output by the semantic module at the t-th time step can be obtained_t. The semantic module may include two coding layers. The network architecture of the position coding module and the semantic module may be similar to the network architecture in fig. 2, and will not be described herein.

According to the character coding scheme, the position information among the characters is enhanced, the dependence of the character recognition result on the semantics is reduced, and therefore the character recognition is more accurate. The character coding scheme provided by the disclosure can be suitable for more complex character recognition scenes, such as recognition of irregular characters, recognition of non-semantic characters and the like, and can also be suitable for scenes such as image recognition and the like, such as image auditing, image analysis and the like.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted.

In addition, the present disclosure also provides an apparatus, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the character recognition methods provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions of the method portions are referred to, and are not described again.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Fig. 4 shows a block diagram of a character recognition apparatus according to an embodiment of the present disclosure, which includes, as shown in fig. 4:

an obtaining module 41, configured to obtain a target image to be identified;

a determining module 42, configured to obtain a character feature of the target image based on the determined position vector and the first image feature of the target image; wherein the position vector is determined based on the position characteristics of the characters in the preset information sequence;

and the recognition module 43 is configured to recognize characters in the target image based on the character features, so as to obtain a character recognition result of the target image.

In a possible implementation manner, the determining module 42 is specifically configured to encode a first image feature of the target image, so as to obtain an encoding result of the first image feature; determining a second image characteristic of the target image according to the encoding result of the first image characteristic; and obtaining character features of the target image based on the determined position vector, the first image features and the second image features.

In a possible implementation manner, the determining module 42 is specifically configured to perform at least one stage of first encoding processing on a plurality of first-dimension feature vectors of the first image feature in sequence to obtain an encoding result of the first image feature.

In a possible implementation manner, the determining module 42 is specifically configured to, for a first encoding process in the at least one first encoding process, sequentially encode input information of the first encoding node by using N first encoding nodes, so as to obtain output results of the N first encoding nodes; under the condition that 1< i is not more than N, the input information of the ith first coding node comprises the output result of the (i-1) th first coding node, and N and i are positive integers; and obtaining the coding result of the first image characteristic according to the output results of the N first coding nodes.

In one possible implementation manner, the input information of the first encoding node further includes a first-dimension feature vector of the first image feature or an output result of a previous-stage first encoding process.

In a possible implementation manner, the determining module 42 is specifically configured to determine an attention weight according to the position vector and the second image feature; and performing feature weighting on the first image features by using the attention weight to obtain the character features of the target image.

In one possible implementation, the apparatus further includes:

the encoding module is used for acquiring a preset information sequence comprising at least one piece of first preset information; and sequentially carrying out at least one stage of second coding processing on the at least one piece of first preset information to obtain the position vector.

In a possible implementation manner, the encoding module is specifically configured to, for a first-level second encoding process in the at least one-level second encoding process, sequentially encode input information of the second encoding node by using M second encoding nodes, and obtain an output result of an M-th second encoding node; under the condition that j is more than 1 and less than or equal to M, the input information of the jth second coding node comprises the output result of the (i-1) th second coding node, and M and j are positive integers; and obtaining the position vector according to the output result of the Mth second coding node.

In a possible implementation manner, the input information of the second encoding node further includes the first preset information or an output result of a previous-stage second encoding process.

In a possible implementation manner, the recognition module 43 is specifically configured to extract semantic features of the target image; and obtaining a character recognition result of the target image based on the semantic features and the character features of the target image.

In a possible implementation manner, the identifying module 43 is specifically configured to sequentially determine semantic features of the target image at least one time step based on the obtained second preset information; and obtaining a character recognition result of the target image at least one time step based on the semantic features and the character features of the target image at least one time step.

In a possible implementation manner, the identifying module 43 is specifically configured to perform at least one stage of third encoding processing on the second preset information to obtain a semantic feature of a first time step in the at least one time step; and performing at least one stage of third coding processing on the character recognition result of the target image at the kth-1 time step to obtain the semantic features of the target image at the kth time step, wherein k is an integer greater than 1.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Fig. 5 is a block diagram illustrating a character recognition apparatus 800 according to an example embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 5, the apparatus 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the apparatus 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power components 806 provide power to the various components of device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 800.

The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed status of the device 800, the relative positioning of components, such as a display and keypad of the device 800, the sensor assembly 814 may also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration/deceleration of the device 800, and a change in the temperature of the device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the apparatus 800 to perform the above-described method.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the above method.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 6 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, the electronic device 1900 may be provided as a server. Referring to fig. 6, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

In an exemplary embodiment, a computer-readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described method.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.