阅读说明：本技术 一种表格创建方法、装置、电子设备及存储介质 (Table creation method and device, electronic equipment and storage medium ) 是由 王洪伟 李长亮 于 2020-05-09 设计创作，主要内容包括：本申请实施例提供了一种表格创建方法、装置、电子设备及存储介质,涉及计算机技术领域。方法包括：获得包含参考表格的图像,作为参考图像；检测参考图像内参考表格包含的表格符,作为目标表格符,获得各个目标表格符的类型和各个目标表格符在参考图像中的位置；根据各个目标表格符的位置和各个目标表格符的指示方向,创建新的表格,其中,新的表格中包含所检测到的目标表格符,且新的表格中各个表格符之间的相对位置与参考表格中各个目标表格符之间的相对位置相同,目标表格符的指示方向为：基于该目标表格符的类型所确定的、表格中该目标表格符与其他目标表格符的连接方向。应用本申请实施例提供的表格创建方案,可以提高创建的表格的准确度。(The embodiment of the application provides a table creating method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: obtaining an image containing a reference table as a reference image; detecting form symbols contained in a reference form in a reference image, taking the form symbols as target form symbols, and obtaining the type of each target form symbol and the position of each target form symbol in the reference image; creating a new table according to the position of each target form character and the indication direction of each target form character, wherein the new table comprises the detected target form characters, the relative position between the form characters in the new table is the same as the relative position between the target form characters in the reference table, and the indication direction of the target form characters is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. By applying the table creation scheme provided by the embodiment of the application, the accuracy of the created table can be improved.)

1. A method of table creation, the method comprising:

obtaining an image containing a reference table as a reference image;

detecting form symbols contained in the reference form in the reference image, taking the form symbols as target form symbols, and obtaining the type of each target form symbol and the position of each target form symbol in the reference image, wherein the form symbols are as follows: preset symbols for forming a table structure;

creating a new table according to the position of each target form character and the indication direction of each target form character, wherein the new table contains the detected target form character, the relative position between the form characters in the new table is the same as the relative position between the target form characters in the reference table, and the indication direction of the target form character is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character.

2. The method according to claim 1, wherein the detecting, as target identifiers, identifiers included in the reference table in the reference image, and obtaining a type of each target identifier and a position of each target identifier in the reference image comprises:

extracting image features of the reference image in a blocking manner;

matching the extracted image features with the features of the preset form symbols, and determining the image area corresponding to the successfully matched image features as the target form symbols;

and obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

3. The method according to claim 1, wherein the detecting, as target identifiers, identifiers included in the reference table in the reference image, and obtaining a type of each target identifier and a position of each target identifier in the reference image comprises:

inputting the reference image into a pre-trained form character detection model, detecting target form characters contained in the reference form in the reference image, and obtaining the type of each target form character output by the form character detection model and the position of each target form character in the reference image, wherein the form character detection model is as follows: and training an initial model of the table symbol detection model by taking a sample image containing a sample table as input and a table symbol of the sample table labeled in advance in the sample image as a label to obtain the network model.

4. The method according to any one of claims 1-3, wherein creating a new table according to the position of each of the target identifiers and the pointing direction of each of the target identifiers comprises:

predicting the structure of the table according to the position of each target form character and the indication direction of each target form character;

and drawing the table of the predicted structure to obtain a new table.

5. The method according to any one of claims 1-3, wherein creating a new table according to the position of each of the target identifiers and the pointing direction of each of the target identifiers comprises:

drawing each detected target form character according to the position of each target form character, wherein the drawn relative position between each target form character is the same as the relative position between each target form character contained in the reference table;

and connecting the target form symbols adjacent to each position along the indication direction of each target form symbol to obtain a new form.

6. The method according to claim 5, wherein said connecting each position-adjacent target form character along the indication direction of each target form character to obtain a new form comprises:

determining a co-located target identifier according to the position of each target identifier, wherein the co-located target identifier is: target table symbols positioned in the same row or the same column;

and connecting the table with target table symbols adjacent to the positions in the co-located target table symbols along the indication direction of each target table symbol in the co-located target table symbols to obtain a new table.

7. The method of claim 6, wherein determining a co-located target identifier based on the location of each of the target identifiers comprises:

detecting horizontal spacing and vertical spacing between the target form symbols according to the positions of the target form symbols;

determining the target table symbol with the horizontal spacing meeting the preset horizontal spacing condition as a co-located target table symbol in the same column;

and determining the target table character with the vertical spacing meeting the preset vertical spacing condition as the co-located target table character in the same row.

8. The method according to claim 5, wherein said connecting each position-adjacent target form character along the indication direction of each target form character to obtain a new form comprises:

connecting the target identifiers adjacent to each other at each position along the indication direction of each target identifier, and deleting the overlapped line segments and/or the error identifiers to obtain a new table, wherein the error identifiers comprise: and the target form character which can not be connected with other target form characters.

9. A method according to any of claims 1-3, wherein after said step of creating a new table according to the position of each of said target identifiers and the indicated direction of each of said target identifiers, the method further comprises:

calculating the coincidence degree of the new table and the reference table in the reference image;

and adjusting the new table according to the contact ratio, updating the new table into an adjusted table, and returning to the step of calculating the contact ratio of the new table and the reference table in the reference image until the contact ratio reaches a preset contact ratio threshold value.

10. A form creation apparatus, the apparatus comprising:

an image obtaining module for obtaining an image containing a reference table as a reference image;

a form character detection module, configured to detect form characters included in the reference form in the reference image, as target form characters, and obtain a type of each target form character and a position of each target form character in the reference image, where the form characters are: preset symbols for forming a table structure;

a table creating module, configured to create a new table according to a position of each target form character and an indication direction of each target form character, where the new table includes the detected target form character, a relative position between each form character in the new table is the same as a relative position between each target form character in the reference table, and an indication direction of each target form character is: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character.

11. The apparatus of claim 10, wherein the form symbol detection module is specifically configured to:

extracting image features of the reference image in a blocking manner;

matching the extracted image features with the features of the preset form symbols, and determining the image area corresponding to the successfully matched image features as the target form symbols;

and obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

12. The apparatus of claim 10, wherein the form symbol detection module is specifically configured to:

inputting the reference image into a pre-trained form character detection model, detecting target form characters contained in the reference form in the reference image, and obtaining the type of each target form character output by the form character detection model and the position of each target form character in the reference image, wherein the form character detection model is as follows: and training an initial model of the table symbol detection model by taking a sample image containing a sample table as input and a table symbol of the sample table labeled in advance in the sample image as a label to obtain the network model.

13. The apparatus according to any of claims 10-12, wherein the table creation module is specifically configured to:

predicting the structure of the table according to the position of each target form character and the indication direction of each target form character;

and drawing the table of the predicted structure to obtain a new table.

14. The apparatus of any of claims 10-12, wherein the table creation module comprises:

a form character drawing unit, configured to draw each of the detected target form characters according to a position of each of the target form characters, where a relative position between the drawn target form characters is the same as a relative position between the target form characters included in the reference table;

and the form character connecting unit is used for connecting the target form characters adjacent to each other along the indication direction of each target form character to obtain a new form.

15. The apparatus of claim 14, wherein the form character connection unit comprises:

a co-located identifier determining subunit, configured to determine a co-located target identifier according to a location of each target identifier, where the co-located target identifier is: target table symbols positioned in the same row or the same column;

and the table character connecting subunit is used for connecting the table characters adjacent to the positions in the co-located target table characters along the indication direction of each co-located target table character in the co-located target table characters to obtain a new table.

16. The apparatus according to claim 15, wherein the co-located identifier determining subunit is specifically configured to:

detecting horizontal spacing and vertical spacing between the target form symbols according to the positions of the target form symbols;

determining the target table symbol with the horizontal spacing meeting the preset horizontal spacing condition as a co-located target table symbol in the same column;

and determining the target table character with the vertical spacing meeting the preset vertical spacing condition as the co-located target table character in the same row.

17. The apparatus according to claim 14, wherein the form symbol connection unit is specifically configured to:

connecting the target identifiers adjacent to each other at each position along the indication direction of each target identifier, and deleting the overlapped line segments and/or the error identifiers to obtain a new table, wherein the error identifiers comprise: and the target form character which can not be connected with other target form characters.

18. The apparatus according to any one of claims 10-12, further comprising:

the coincidence degree calculation module is used for calculating the coincidence degree of the new table and the reference table in the reference image after the new table is created;

and the table adjusting module is used for adjusting the new table according to the contact ratio, updating the new table into an adjusted table, and triggering the contact ratio calculating module until the contact ratio reaches a preset contact ratio threshold value.

19. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-9 when executing a program stored in the memory.

20. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-9.

Technical Field

The present application relates to the field of computer technologies, and in particular, to a table creating method and apparatus, an electronic device, and a storage medium.

Background

In an office scenario, it is often necessary to refer to an existing form and create a new form, where the form for reference may be in a picture format.

In the prior art, when a new table is created with reference to a table in a picture format, it is usually necessary to detect line segments in a picture by using an edge detection technique, obtain information such as length, position, and direction of each line segment in the picture, and draw each line segment according to the obtained information, thereby creating and obtaining the new table.

Although a new table can be created by applying the prior art, since the picture may also contain other contents, such as characters, graphics, and the like, when a line segment in the picture is detected, the line segment is shielded, interfered, and the like by the other contents, so that the accuracy of information such as the length and the position of the detected line segment is low, and further, the accuracy of the created table is low.

Disclosure of Invention

An object of the embodiments of the present application is to provide a table creating method, an apparatus, an electronic device, and a storage medium, so as to improve accuracy of a created table. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a table creating method, where the method includes:

obtaining an image containing a reference table as a reference image;

detecting form symbols contained in the reference form in the reference image, taking the form symbols as target form symbols, and obtaining the type of each target form symbol and the position of each target form symbol in the reference image, wherein the form symbols are as follows: preset symbols for forming a table structure;

creating a new table according to the position of each target form character and the indication direction of each target form character, wherein the new table contains the detected target form character, the relative position between the form characters in the new table is the same as the relative position between the target form characters in the reference table, and the indication direction of the target form character is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character.

In an embodiment of the application, the detecting, as target identifiers, identifiers included in the reference table in the reference image, and obtaining a type of each of the target identifiers and a position of each of the target identifiers in the reference image includes:

extracting image features of the reference image in a blocking manner;

matching the extracted image features with the features of the preset form symbols, and determining the image area corresponding to the successfully matched image features as the target form symbols;

and obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

In an embodiment of the application, the detecting, as target identifiers, identifiers included in the reference table in the reference image, and obtaining a type of each of the target identifiers and a position of each of the target identifiers in the reference image includes:

inputting the reference image into a pre-trained form character detection model, detecting target form characters contained in the reference form in the reference image, and obtaining the type of each target form character output by the form character detection model and the position of each target form character in the reference image, wherein the form character detection model is as follows: and training an initial model of the table symbol detection model by taking a sample image containing a sample table as input and a table symbol of the sample table labeled in advance in the sample image as a label to obtain the network model.

In an embodiment of the application, the creating a new table according to the position of each of the target identifiers and the indication direction of each of the target identifiers includes:

predicting the structure of the table according to the position of each target form character and the indication direction of each target form character;

and drawing the table of the predicted structure to obtain a new table.

In an embodiment of the application, the creating a new table according to the position of each of the target identifiers and the indication direction of each of the target identifiers includes:

drawing each detected target form character according to the position of each target form character, wherein the drawn relative position between each target form character is the same as the relative position between each target form character contained in the reference table;

and connecting the target form symbols adjacent to each position along the indication direction of each target form symbol to obtain a new form.

In an embodiment of the application, the connecting the target identifiers adjacent to each other in each position along the indication direction of each target identifier to obtain a new table includes:

determining a co-located target identifier according to the position of each target identifier, wherein the co-located target identifier is: target table symbols positioned in the same row or the same column;

and connecting the table with target table symbols adjacent to the positions in the co-located target table symbols along the indication direction of each target table symbol in the co-located target table symbols to obtain a new table.

In an embodiment of the present application, the determining the co-located target identifiers according to the positions of the target identifiers includes:

detecting horizontal spacing and vertical spacing between the target form symbols according to the positions of the target form symbols;

determining the target table symbol with the horizontal spacing meeting the preset horizontal spacing condition as a co-located target table symbol in the same column;

and determining the target table character with the vertical spacing meeting the preset vertical spacing condition as the co-located target table character in the same row.

In an embodiment of the application, the connecting the target identifiers adjacent to each other in each position along the indication direction of each target identifier to obtain a new table includes:

connecting the target identifiers adjacent to each other at each position along the indication direction of each target identifier, and deleting the overlapped line segments and/or the error identifiers to obtain a new table, wherein the error identifiers comprise: and the target form character which can not be connected with other target form characters.

In an embodiment of the application, after the step of creating a new table according to the position of each of the target identifiers and the indication direction of each of the target identifiers, the method further includes:

calculating the coincidence degree of the new table and the reference table in the reference image;

and adjusting the new table according to the contact ratio, updating the new table into an adjusted table, and returning to the step of calculating the contact ratio of the new table and the reference table in the reference image until the contact ratio reaches a preset contact ratio threshold value.

In a second aspect, an embodiment of the present application provides a table creating apparatus, where the apparatus includes:

an image obtaining module for obtaining an image containing a reference table as a reference image;

a form character detection module, configured to detect form characters included in the reference form in the reference image, as target form characters, and obtain a type of each target form character and a position of each target form character in the reference image, where the form characters are: preset symbols for forming a table structure;

a table creating module, configured to create a new table according to a position of each target form character and an indication direction of each target form character, where the new table includes the detected target form character, a relative position between each form character in the new table is the same as a relative position between each target form character in the reference table, and an indication direction of each target form character is: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character.

In an embodiment of the application, the form symbol detection module is specifically configured to:

extracting image features of the reference image in a blocking manner;

matching the extracted image features with the features of the preset form symbols, and determining the image area corresponding to the successfully matched image features as the target form symbols;

and obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

In an embodiment of the application, the form symbol detection module is specifically configured to:

inputting the reference image into a pre-trained form character detection model, detecting target form characters contained in the reference form in the reference image, and obtaining the type of each target form character output by the form character detection model and the position of each target form character in the reference image, wherein the form character detection model is as follows: and training an initial model of the table symbol detection model by taking a sample image containing a sample table as input and a table symbol of the sample table labeled in advance in the sample image as a label to obtain the network model.

In an embodiment of the application, the table creation module is specifically configured to:

predicting the structure of the table according to the position of each target form character and the indication direction of each target form character;

and drawing the table of the predicted structure to obtain a new table.

In an embodiment of the application, the table creating module includes:

a form character drawing unit, configured to draw each of the detected target form characters according to a position of each of the target form characters, where a relative position between the drawn target form characters is the same as a relative position between the target form characters included in the reference table;

and the form character connecting unit is used for connecting the target form characters adjacent to each other along the indication direction of each target form character to obtain a new form.

In an embodiment of the present application, the form character connection unit includes:

a co-located identifier determining subunit, configured to determine a co-located target identifier according to a location of each target identifier, where the co-located target identifier is: target table symbols positioned in the same row or the same column;

and the table character connecting subunit is used for connecting the table characters adjacent to the positions in the co-located target table characters along the indication direction of each co-located target table character in the co-located target table characters to obtain a new table.

In an embodiment of the present application, the parity table symbol determining subunit is specifically configured to:

detecting horizontal spacing and vertical spacing between the target form symbols according to the positions of the target form symbols;

determining the target table symbol with the horizontal spacing meeting the preset horizontal spacing condition as a co-located target table symbol in the same column;

and determining the target table character with the vertical spacing meeting the preset vertical spacing condition as the co-located target table character in the same row.

In an embodiment of the application, the form character connection unit is specifically configured to:

connecting the target identifiers adjacent to each other at each position along the indication direction of each target identifier, and deleting the overlapped line segments and/or the error identifiers to obtain a new table, wherein the error identifiers comprise: and the target form character which can not be connected with other target form characters.

In one embodiment of the present application, the apparatus further comprises:

the coincidence degree calculation module is used for calculating the coincidence degree of the new table and the reference table in the reference image after the new table is created;

and the table adjusting module is used for adjusting the new table according to the contact ratio, updating the new table into an adjusted table, and triggering the contact ratio calculating module until the contact ratio reaches a preset contact ratio threshold value.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of the first aspect when executing a program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of any one of the first aspect.

Embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any of the above-described table creation methods.

The embodiment of the application has the following beneficial effects:

when a new table is created by applying the scheme provided by the embodiment of the application, an image containing a reference table can be obtained and used as a reference image, the identifiers contained in the reference table in the reference image are detected and used as target identifiers, the types of the target identifiers and the positions of the target identifiers in the reference image are obtained, and the new table is created according to the positions of the target identifiers and the indication directions of the target identifiers. Wherein, the indication direction of the target form symbol is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. The new table is created to include the detected target identifiers, and the relative position between the target identifiers in the new table is the same as the relative position between the target identifiers in the table. Thus, the target form character of the reference form in the reference image is detected, and a new form is created according to the position and type of the detected target form character. Because the form character is not easy to be shielded and interfered by other contents in the reference image, the accuracy of the target form character of the reference form in the detected reference image is higher, and the accuracy of the new form created according to the target form character is higher. Therefore, by applying the table creation scheme provided by the embodiment of the application, the accuracy of the created table can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of a table creating method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a reference image according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a table symbol according to an embodiment of the present application;

fig. 4 is a schematic flowchart of a method for detecting a form symbol according to an embodiment of the present application;

fig. 5 is a schematic flowchart of a model training method according to an embodiment of the present application;

fig. 6 is a flowchart illustrating another table creating method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a coincident line segment provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of a table creation apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to improve the accuracy of the created table, embodiments of the present application provide a table creating method, an apparatus, an electronic device, and a storage medium, which are described in detail below.

Referring to fig. 1, fig. 1 is a schematic flowchart of a table creation method provided in an embodiment of the present application. The form creation method can be applied to electronic equipment such as mobile phones, tablet computers and electronic computers, and can also be applied to Office software clients such as Office and WPS. The table creation method includes the following steps 101 to 103.

In step 101, an image containing a reference table is obtained as a reference image.

The reference table may be a table such as an invoice, a bill, a registration table, etc., and may be a reference of a newly created table.

Specifically, the reference table may be subjected to image acquisition by an image acquisition device to obtain a reference image, and the reference image may be an image containing the reference table obtained from a database. Referring to fig. 2, fig. 2 is a schematic view of a reference image provided in an embodiment of the present application, and a table of an invoice is used as a reference table, and the reference image shown in fig. 2 can be obtained by performing image acquisition on the invoice. It should be noted that fig. 2 is only for illustrating the image containing the reference table, and the specific content of each text in the figure has no exemplary effect on the embodiment of the present application.

In an embodiment of the application, after the reference image is obtained, the reference image can be subjected to image enhancement, noise reduction and other processing, so that the reference table in the reference image can be clearer, and the subsequent detection of the reference table is facilitated. In addition, the reference image can be zoomed, rotated, stretched and the like, so that the reference table in the reference image is in a horizontal position and is not deformed, and the accuracy of detecting the form symbols in the reference table in the follow-up process can be improved.

Step 102, detecting the form symbols contained in the reference form in the reference image as target form symbols, and obtaining the type of each target form symbol and the position of each target form symbol in the reference image.

Wherein, the form symbol is: the preset symbols for forming the table structure, the type of the table symbol may include: the cross form table symbol, the T form table symbol, the L form table symbol,A shape form symbol, a "" L "" form symbol, and the like. Referring to fig. 3, fig. 3 is a schematic diagram of a form provided in the present application, and as shown in the diagram, 301 is a "ten" form, 302 is a "T" form, and 303 is an "L" form. It should be noted that fig. 3 is only for illustrating the image containing the reference table, and the specific content of each text in the figure has no exemplary effect on the embodiment of the present application.

Specifically, the form symbols included in the reference form in the reference image may be detected, the detected form symbols are used as target form symbols, and the type of each target form symbol and the position of each target form symbol in the reference image are detected.

Step 103, creating a new table according to the position of each target form character and the indication direction of each target form character.

Wherein the new table contains the detected target form character. Specifically, the target identifiers included in the new table are consistent in number and same in type as the detected target identifiers. For example, if the detected target form includes 5 "ten" form forms and 7 "T" form forms, the created new form also includes 5 "ten" form forms and 7 "T" form forms.

And the relative position between the identifiers in the new table is the same as the relative position between the target identifiers in the reference table. For example, if the two detected target identifiers are located in the same column in the reference image and the distance is 30 pixels, the two detected target identifiers in the created new table are also located in the same column and the distance is 30 pixels.

In addition, the ratio of the spacing distance between the target characters in the new table to the length and width of the new table may be the same as the ratio of the spacing distance between the target characters in the reference table to the length and width of the reference table. For example, in the interval distance between two target form symbols in the reference form, the horizontal distance accounts for 20% of the length of the reference form, and the vertical distance accounts for 10% of the width of the reference form; then the horizontal distance also accounts for 20% of the length of the new form and the vertical distance also accounts for 10% of the width of the new form, of the separation distance between two target form characters in the new form. The ratio of the spacing distance between the target characters in the new table to the length and width of the new table may be the same as the ratio of the spacing distance between the target characters in the reference table to the length and width of the reference image.

The indication direction of the target form symbol is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. The type of the target form character has a corresponding relation with the indication direction. For example, a "ten" form designator, when connected to other target form designators, may be connected up, left, right, down, and thus, the indicated direction of the "ten" form designator includes up, left, right, and down; the "T" shaped form may be connected to other target forms in a left, right, and downward direction, and thus the direction indicated by the "T" shaped form includes a left, right, and downward direction.

Specifically, the indication direction of each target form character can be determined according to the type of each target form character, and the connection relationship between different target form characters can be determined according to the indication direction of each target form character according to the position of each target form character, so that a new form can be created. This may allow the new table created to be close to the reference table in the reference image, improving the accuracy of the created table.

When a new table is created by applying the scheme provided by the above embodiment, an image containing a reference table can be obtained as a reference image, form symbols contained in the reference table in the reference image are detected as target form symbols, the type of each target form symbol and the position of each target form symbol in the reference image are obtained, and the new table is created according to the position of each target form symbol and the indication direction of each target form symbol. Wherein, the indication direction of the target form symbol is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. The new table is created to include the detected target identifiers, and the relative position between the target identifiers in the new table is the same as the relative position between the target identifiers in the table. Thus, the target form character of the reference form in the reference image is detected, and a new form is created according to the position and type of the detected target form character. Because the form character is not easy to be shielded and interfered by other contents in the reference image, the accuracy of the target form character of the reference form in the detected reference image is higher, and the accuracy of the new form created according to the target form character is higher. Therefore, by applying the table creation scheme provided by the above embodiment, the accuracy of the created table can be improved.

Referring to fig. 4, fig. 4 is a flowchart illustrating a method for detecting a glyph according to an embodiment of the present application, and the method may specifically include the following steps 401 and 403 when detecting the glyph included in the reference table in the reference image in the step 102.

Step 401, extracting image features of the reference image in blocks.

Specifically, the reference image may be partitioned according to a preset number of partitions, which may be 200, 300, 500, and the like, and the reference image is equally partitioned into a number of image blocks, where the size and the shape of each image block are the same. The shape of each image block may be square, rectangular, etc. The reference image may also be partitioned according to a preset image block size, which may be 5 × 5, 3 × 3, 5 × 7, and so on.

After obtaining a plurality of image blocks, the image features of each image block may be extracted, where the image features may be texture features, line features, and the like of each image block. In the embodiment of the application, the edge detection operator may be used to extract image features of the image block, such as a sobel operator, a laplacian operator, and the like, and the neural network model may also be used to extract image features of the image block.

And step 402, matching the extracted image features with the features of the preset form character, and determining the image area corresponding to the successfully matched image features as the target form character.

Specifically, for each image block, feature matching may be performed on the image features of the image block and features of different types of form symbols in sequence, if matching is successful, it is indicated that an image area corresponding to the image features is a target form symbol, and if matching is unsuccessful, feature matching is performed on the next type of form symbol until all types of form symbols are traversed.

In one embodiment of the present application, when performing feature matching, a similarity between an image feature and a feature of a form character may be calculated, and when the similarity reaches a preset similarity threshold, the matching may be considered to be successful.

Step 403, obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

Specifically, for the determined target form character, the type of the target form character is as follows: the type of form character successfully matched in step 402. For example, assuming that an image block is successfully matched with a "ten" shape character, the type of the target character of the image area corresponding to the image block is "ten". And the position of the image area corresponding to the image block in the reference image can be obtained as the position of the target form character in the reference image.

In this way, the form character in the reference image is detected in a characteristic matching mode, and the accuracy of the detected target form character can be improved.

In an embodiment of the application, for the step 102, when detecting the form character in the reference image, specifically, the reference image may be input into a form character detection model trained in advance, a target form character included in a reference form in the reference image is detected, and a type of each target form character and a position of each target form character output by the form character detection model in the reference image are obtained.

Wherein, the form symbol detection model is as follows: and training an initial model of the table symbol detection model by taking the sample image containing the sample table as input and the table symbol of the sample table labeled in advance in the sample image as label. The form symbol detection model may be a convolutional neural network model, a cyclic neural network model, a deep neural network model, or the like.

The form character detection model can realize the scheme of the embodiment of fig. 4, detect the form characters in the reference image in a feature matching manner, specifically extract the image features of the reference image in blocks, perform feature matching on each image feature with different types of form characters in sequence, determine the target form characters based on the matching results, further obtain the positions and types of the target form characters, and output the results.

Referring to fig. 5, fig. 5 is a schematic flowchart of a model training method according to an embodiment of the present application, where the training method includes the following steps 501 to 504.

Step 501, sample images are obtained, and the form symbols of the forms in each sample image are labeled.

The sample image may be a plurality of images obtained by acquiring images of a plurality of different types of tables, or may be a plurality of images including tables obtained from a database.

In one embodiment of the present application, the staff member can manually label the form symbols of the form in the sample image, including labeling the position and type of each form symbol.

Step 502, inputting each sample image into the initial model to obtain an output result.

Specifically, each sample image is sequentially input into an initial model, a form character in each sample image is detected by the initial model, and the position and type of each form character are output as an output result.

Step 503, calculating the loss between each output result and the labeled form character.

Specifically, the loss between each output result and the labeled table symbol may be calculated by using a preset loss function, which may be a cross entropy loss function, a hinge loss function, an exponential loss function, or the like.

Step 504, adjusting the parameters of the initial model based on the calculated loss, updating the initial model into a model with the adjusted parameters, and returning to step 502 until the training end condition is met.

The larger the loss value is, the larger the difference between the output result of the initial model and the labeled form symbol is, that is, the lower the accuracy of the form symbol detection performed by the initial model is, so that it is necessary to perform parameter adjustment on the initial model and train the model after parameter adjustment again until the training end condition is satisfied.

In an embodiment of the application, the training end condition may be that the training number reaches a preset number threshold, where the number threshold may be 5000 times, 10000 times, 50000 times, and the like, and when the training number reaches the preset number threshold, the model may be considered to be converged, so that the training is ended. The training end condition may be that the loss between each output result and the labeled form symbol is less than a preset loss threshold.

In an embodiment of the application, for step 103, when creating a new table, the structure of the table may be predicted according to the position of each target identifier and the indication direction of each target identifier, and the table with the predicted structure may be drawn to obtain the new table.

Wherein, the structure of the table includes: the length, height of the table, and the length, height, location, etc. of the cells contained within the table.

Specifically, since the target identifiers are symbols constituting the table structure, and the indication direction of the target identifiers can be determined according to the type of the target identifiers, the connection relationship between the target identifiers can be further determined according to the indication direction and the position of each target identifier. The connection relation includes: the direction, position, length, etc. of the connecting line segment for connecting the respective form characters. For example, assuming that two "ten" form identifiers are located in the same column and the interval is 20 pixels, it may be determined that the direction of a connecting line segment for connecting the two form identifiers is the vertical direction, the start point and the end point of the connecting line segment are the two "ten" form identifiers, respectively, and the length of the line segment is 20 pixels. Alternatively, the length in the above connection relationship may be a ratio of the length of the connection line segment to the length and width of the reference image. For example, assuming that two "ten" form identifiers are located in the same column and the interval occupies 20% of the width of the reference image, it is determined that the direction of the connecting line segment for connecting the two identifiers is the vertical direction, the start point and the end point of the connecting line segment are the two "ten" form identifiers, respectively, and the length of the line segment in the new table occupies 20% of the width of the new table.

From the connection relationship, the length, height, and position of the table, and the cells included in the table, can be further predicted. In an embodiment of the present application, the length of the table may be obtained by adding the lengths of the connecting line segments in the horizontal direction, the position of which is at the bottom or the top, and the length of the connecting line segments in the vertical direction, the position of which is at the left or the right, may be added to obtain the height of the table. And searching four line segments connected end to end in pairs according to the length, the position and the direction of each connecting line segment, wherein the graph formed by the four line segments is the cell. According to the lengths and the positions of the four line segments, the position, the length and the height of the cell can be obtained. The length, height, and position of each cell included in the table can be determined by traversing all the connecting line segments.

The structure of the table can be predicted by using the determined length and height of the table and the length, height and position of the cells contained in the table. And drawing the table of the predicted structure to obtain a new table.

Referring to fig. 6, fig. 6 is a flowchart illustrating another table creating method provided in the embodiment of the present application, and for the step 103, the method for creating a new table includes the following steps 1031 to 1032.

Step 1031, drawing each detected target form character according to the position of each target form character.

And the drawn relative positions of the target identifiers are the same as the relative positions of the target identifiers contained in the reference table.

Specifically, the detected target identifiers may be drawn, and the drawn target identifiers and the detected target identifiers may be kept consistent in number and same in type. For example, assuming that the detected target identifiers include 5 "ten" shape identifiers and 7 "T" shape identifiers, 5 "ten" shape identifiers and 7 "T" shape identifiers are correspondingly drawn. And the relative positions between the drawn target identifiers are the same as the relative positions between the target identifiers in the reference table. For example, if two detected target identifiers are located in the same column in the reference image and the distance between the two detected target identifiers is 30 pixels, the two drawn target identifiers are also located in the same column and the distance between the two drawn target identifiers is 30 pixels. Alternatively, assuming that the two detected target identifiers are located in the same column in the reference image and the space is 20% of the width of the reference image, the two drawn target identifiers are also located in the same column and the space is 20% of the width of the new table.

And 1032, connecting the target identifiers adjacent to each other in each position along the indication direction of each target identifier to obtain a new table.

Specifically, for each target character, the direction of the target character may be referred to, and the target characters adjacent to each other in the direction of the direction may be connected. For example, assuming that the target character is an "L" -shaped character, the pointing direction of the character is upward and rightward, and thus the target characters adjacent above the target character are connected, and the target characters adjacent to the right of the target character are connected. And traversing each target form character in sequence, and connecting each target form character to obtain a new form.

In an embodiment of the present application, in step 1032, when the target identifiers are connected, the co-located target identifiers may be determined according to positions of the target identifiers, and the target identifiers adjacent to the positions in the co-located target identifiers are connected along the indication direction of the target identifiers in the co-located target identifiers, so as to obtain a new table.

Wherein, the co-located target table symbol is: and target table characters located in the same row or the same column.

Specifically, the drawn target identifiers may be first classified, the target identifiers located in the same row or the same column are used as the parity identifiers, and then the respective parity identifiers are connected according to the positions and the indication directions of the respective target identifiers in the parity identifiers. Therefore, the target form symbols positioned in the same row or the same column are connected, so that the connected line segments are not easy to incline, and the form creating effect is improved.

In an embodiment of the present application, when determining the parity identifiers, the horizontal spacing and the vertical spacing between the target identifiers may be detected according to the positions of the target identifiers, the target identifiers whose horizontal spacing satisfies the preset horizontal spacing condition are determined as the parity target identifiers located in the same column, and the target identifiers whose vertical spacing satisfies the preset vertical spacing condition are determined as the parity target identifiers located in the same row.

The above-mentioned horizontal pitch condition may be: the horizontal spacing of the target form character in the horizontal direction is not more than a preset horizontal spacing threshold, and the following steps can be also included: the proportion of the horizontal spacing of the target form character in the horizontal direction to the length of the form is not more than a preset horizontal proportion threshold value.

Specifically, the horizontal distances of the drawn target identifiers in the horizontal direction may be calculated two by two, and if the horizontal distance is not greater than a preset horizontal distance threshold, the horizontal positions of the two target identifiers may be considered to be the same, so that the two target identifiers may be determined to be the co-located target identifiers located in the same column. The horizontal distance threshold may be 2 pixels, 3 pixels, 5 pixels, and the like. The ratio of the horizontal distance of the drawn target form symbols in the horizontal direction to the form length may also be calculated two by two, and if the ratio is not greater than a preset horizontal ratio threshold, the two target form symbols may be determined as the co-located target form symbols located in the same column.

The above-mentioned vertical pitch condition may be: the vertical spacing of the target form character in the vertical direction is not more than a preset vertical spacing threshold, and the following steps can be taken: the proportion of the vertical spacing of the target form character in the vertical direction to the width of the form is not more than a preset vertical proportion threshold value.

Specifically, the vertical distances of the drawn target identifiers in the vertical direction may be calculated two by two, and if the vertical distance is not greater than a preset vertical distance threshold, it may be considered that the vertical positions of the two target identifiers are the same, so that the two target identifiers may be determined as the co-located target identifiers located in the same row. The horizontal distance threshold may be 2 pixels, 3 pixels, 5 pixels, and the like. The ratio of the vertical spacing of the drawn target form symbols in the vertical direction to the form width may also be calculated two by two, and if the ratio is not greater than a preset vertical ratio threshold, the two target form symbols may be determined as the co-located target form symbols located in the same row.

In one embodiment of the present application, to facilitate the subsequent connection of the parity symbols, the position of the parity symbols may be corrected by adjusting the horizontal spacing between the parity symbols in the same column to 0 and adjusting the vertical spacing between the parity symbols in the same row to 0.

In an embodiment of the application, in step 1032, when the target identifiers are connected to obtain a new table, the target identifiers adjacent to each other at each position may be connected along the indication direction of each target identifier, and the overlapped line segments and/or the error identifiers are deleted to obtain the new table.

Referring to fig. 7, fig. 7 is a schematic diagram of a coincident line segment provided in an embodiment of the present application, where 701 denotes a first target form symbol, 702 denotes a second target form symbol, and a connection line segment can be obtained when the first target form symbol 701 is connected with the second target form symbol 702; when the second target identifier 702 is connected to the first target identifier 701, a line segment connecting the first target identifier 701 and the second target identifier 702 may be obtained again, so as to cause a coincident line segment to appear in the table, which may affect the creation effect of the table.

The error table symbol includes: and the target form character which can not be connected with other target form characters. Specifically, when there is no other target character in all the directions indicated by a target character, the target character is difficult to be connected to another target character, and thus the target character is regarded as an error character.

Specifically, when the table is created, the target form symbols are connected first, and then the overlapped line segments and the error form symbols are deleted, so that a new table is obtained, and the creation effect of the table can be improved.

In an embodiment of the application, after the new table is created, the overlap ratio of the new table and the reference table in the reference image may be further calculated, the new table is adjusted according to the overlap ratio, the new table is updated to the adjusted table, and the step of calculating the overlap ratio of the new table and the reference table in the reference image is returned until the overlap ratio reaches the preset overlap ratio threshold.

Specifically, whether the new table is close to the reference table or not can be judged by calculating the contact ratio, and the higher the contact ratio is, the closer the created new table is to the reference table is; the lower the degree of coincidence, the greater the difference between the created new table and the reference table. When the contact ratio does not reach the contact ratio threshold value, the difference between the new table and the reference table is large, and the new table needs to be adjusted until the contact ratio reaches the contact ratio threshold value.

In one embodiment of the present application, the coincidence degree of the new table and the reference table in the reference image can be determined by calculating the similarity. The new table and the reference table in the reference image can be converted into the same coordinate system, whether line segments, table symbols and the like in the new table and the reference table are overlapped or not is judged, the number of the overlapped line segments and table symbols is counted, and the ratio of the number to the total number of the line segments and the table symbols in the new table is calculated to serve as the overlap ratio.

The accuracy of the created table can be further improved by adjusting the created table with the degree of overlap.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a table creating apparatus provided in an embodiment of the present application, where the apparatus includes:

an image obtaining module 801, configured to obtain an image including a reference table as a reference image;

a form character detection module 802, configured to detect form characters included in the reference form in the reference image, as target form characters, and obtain a type of each target form character and a position of each target form character in the reference image, where the form characters are: preset symbols for forming a table structure;

a table creating module 803, configured to create a new table according to the position of each target identifier and the indication direction of each target identifier, where the new table includes the detected target identifier, and the relative position between each identifier in the new table is the same as the relative position between each target identifier in the reference table, and the indication direction of each target identifier is: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character.

In an embodiment of the present application, the form symbol detection module 802 is specifically configured to:

extracting image features of the reference image in a blocking manner;

matching the extracted image features with the features of the preset form symbols, and determining the image area corresponding to the successfully matched image features as the target form symbols;

and obtaining the type of the determined target form character as the type of the matched form character, and obtaining the position of the image area corresponding to the determined target form character in the reference image as the position of the determined target form character in the reference image.

In an embodiment of the present application, the form symbol detection module 802 is specifically configured to:

inputting the reference image into a pre-trained form character detection model, detecting target form characters contained in the reference form in the reference image, and obtaining the type of each target form character output by the form character detection model and the position of each target form character in the reference image, wherein the form character detection model is as follows: and training an initial model of the table symbol detection model by taking a sample image containing a sample table as input and a table symbol of the sample table labeled in advance in the sample image as a label to obtain the network model.

In an embodiment of the present application, the table creating module 803 is specifically configured to:

predicting the structure of the table according to the position of each target form character and the indication direction of each target form character;

and drawing the table of the predicted structure to obtain a new table.

In an embodiment of the present application, the table creating module 803 includes:

a form character drawing unit, configured to draw each of the detected target form characters according to a position of each of the target form characters, where a relative position between the drawn target form characters is the same as a relative position between the target form characters included in the reference table;

and the form character connecting unit is used for connecting the target form characters adjacent to each other along the indication direction of each target form character to obtain a new form.

In an embodiment of the present application, the form character connection unit includes:

a co-located identifier determining subunit, configured to determine a co-located target identifier according to a location of each target identifier, where the co-located target identifier is: target table symbols positioned in the same row or the same column;

and the table character connecting subunit is used for connecting the table characters adjacent to the positions in the co-located target table characters along the indication direction of each co-located target table character in the co-located target table characters to obtain a new table.

In an embodiment of the present application, the parity table symbol determining subunit is specifically configured to:

detecting horizontal spacing and vertical spacing between the target form symbols according to the positions of the target form symbols;

determining the target table symbol with the horizontal spacing meeting the preset horizontal spacing condition as a co-located target table symbol in the same column;

and determining the target table character with the vertical spacing meeting the preset vertical spacing condition as the co-located target table character in the same row.

In an embodiment of the application, the form character connection unit is specifically configured to:

connecting the target identifiers adjacent to each other at each position along the indication direction of each target identifier, and deleting the overlapped line segments and/or the error identifiers to obtain a new table, wherein the error identifiers comprise: and the target form character which can not be connected with other target form characters.

In one embodiment of the present application, the apparatus further comprises:

the coincidence degree calculation module is used for calculating the coincidence degree of the new table and the reference table in the reference image after the new table is created;

and the table adjusting module is used for adjusting the new table according to the contact ratio, updating the new table into an adjusted table, and triggering the contact ratio calculating module until the contact ratio reaches a preset contact ratio threshold value.

When a new table is created by applying the scheme provided by the above embodiment, an image containing a reference table can be obtained as a reference image, form symbols contained in the reference table in the reference image are detected as target form symbols, the type of each target form symbol and the position of each target form symbol in the reference image are obtained, and the new table is created according to the position of each target form symbol and the indication direction of each target form symbol. Wherein, the indication direction of the target form symbol is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. The new table is created to include the detected target identifiers, and the relative position between the target identifiers in the new table is the same as the relative position between the target identifiers in the table. Thus, the target form character of the reference form in the reference image is detected, and a new form is created according to the position and type of the detected target form character. Because the form character is not easy to be shielded and interfered by other contents in the reference image, the accuracy of the target form character of the reference form in the detected reference image is higher, and the accuracy of the new form created according to the target form character is higher. Therefore, by applying the table creation scheme provided by the above embodiment, the accuracy of the created table can be improved.

The embodiment of the present application further provides an electronic device, as shown in fig. 9, which includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904,

a memory 903 for storing computer programs;

the processor 901 is configured to implement the steps of the table creation method when executing the program stored in the memory 903.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In yet another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above table creation methods.

In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the above-described table creation methods.

When a new table is created by applying the scheme provided by the above embodiment, an image containing a reference table can be obtained as a reference image, form symbols contained in the reference table in the reference image are detected as target form symbols, the type of each target form symbol and the position of each target form symbol in the reference image are obtained, and the new table is created according to the position of each target form symbol and the indication direction of each target form symbol. Wherein, the indication direction of the target form symbol is as follows: and the connection direction of the target form character and other target form characters in the form is determined based on the type of the target form character. The new table is created to include the detected target identifiers, and the relative position between the target identifiers in the new table is the same as the relative position between the target identifiers in the table. Thus, the target form character of the reference form in the reference image is detected, and a new form is created according to the position and type of the detected target form character. Because the form character is not easy to be shielded and interfered by other contents in the reference image, the accuracy of the target form character of the reference form in the detected reference image is higher, and the accuracy of the new form created according to the target form character is higher. Therefore, by applying the table creation scheme provided by the above embodiment, the accuracy of the created table can be improved.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are substantially similar to method embodiments and therefore are described with relative ease, as appropriate, with reference to the partial description of the method embodiments.

The above description is only for the preferred embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.