Method, system, storage medium and terminal for constructing logical structure tree reported by listed companies

文档序号：1831765 发布日期：2021-11-12 浏览：18次中文

阅读说明：本技术 上市公司报告的逻辑结构树构建方法、系统、存储介质及终端 (Method, system, storage medium and terminal for constructing logical structure tree reported by listed companies ) 是由何思佳孙峥于 2021-08-10 设计创作，主要内容包括：本发明提供一种上市公司报告的逻辑结构树构建方法、系统、存储介质及终端,包括以下步骤：将上市公司报告转换为html格式文档；对所述html格式文档进行预处理,获取html层级结构文档；获取逻辑结构树的预设结构,在所述html层级结构文档中抽取所述预设结构的各级节点；基于所述各级节点,基于文本匹配的方式提取匹配内容以获取所述逻辑结构树。本发明的上市公司报告的逻辑结构树构建方法、系统、存储介质及终端基于上市公司报告的层级结构,提取特定节点信息来构建逻辑结构树,实现了上市公司报告信息的有效处理。(The invention provides a method, a system, a storage medium and a terminal for constructing a logical structure tree reported by a listed company, which comprise the following steps: converting the listed company report into a html format document; preprocessing the html-format document to obtain an html hierarchical structure document; acquiring a preset structure of a logic structure tree, and extracting nodes of each level of the preset structure from the html hierarchical structure document; and extracting matching contents based on the nodes of each level in a text matching mode to obtain the logic structure tree. The method, the system, the storage medium and the terminal for constructing the logical structure tree of the listed company report extract the specific node information to construct the logical structure tree based on the hierarchical structure of the listed company report, thereby realizing the effective processing of the listed company report information.)

1. A method for constructing a logical structure tree reported by listed companies is characterized by comprising the following steps: the method comprises the following steps:

converting the listed company report into a html format document;

preprocessing the html-format document to obtain an html hierarchical structure document;

acquiring a preset structure of a logic structure tree, and extracting nodes of each level of the preset structure from the html hierarchical structure document;

and extracting matching contents based on the nodes of each level in a text matching mode to obtain the logic structure tree.

2. The method of constructing a logical structure tree of a listed company report as claimed in claim 1 wherein: converting the listing company report to an html format document includes the steps of:

acquiring a listed company report in a PDF format;

and converting the PDF-format listed company report into an html-format document, wherein the html-format document retains all text contents and format information of the PDF-format listed company report.

3. The method of constructing a logical structure tree of a listed company report as claimed in claim 1 wherein: preprocessing the html format document, and acquiring the html hierarchical structure document comprises the following steps:

identifying a directory of the html-format document;

and deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

4. The method of constructing a logical structure tree of a listed company report as claimed in claim 1 wherein: the method for extracting the nodes of the preset structure from the html hierarchical structure document comprises the following steps:

identifying a hierarchical structure of the html hierarchically structured document;

based on the hierarchical structure, recursively searching all directly subordinate child nodes of the preset structure by adopting a strategy of from top to bottom, and from the top to the bottom;

after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes;

and analogizing in turn until the next-level node cannot be searched in the coverage area.

5. The method of constructing a logical structure tree of a listed company report as claimed in claim 1 wherein: based on the nodes at each level, extracting matching contents based on a text matching mode to obtain the logic structure tree comprises the following steps:

verifying the logic structure tree based on the html document directory;

after the verification is passed, column searching is carried out; and matching corresponding column objects on the subtrees of the logic structure tree, wherein the column objects inherit from the subtree objects until the column searching of the logic structure tree is finished.

6. The method of constructing a logical structure tree of a listed company report as claimed in claim 5 wherein: when column searching is carried out, identifying a table in the column; identifying the table in the column includes the steps of:

extracting the header of the table and carrying out alias standardization;

ordering the list head after the alias standardization according to the first letter to generate a character string;

converting the character string into MD5 value, and performing table identification based on the MD5 value.

7. The method of constructing a logical structure tree of a listed company report as claimed in claim 5 wherein: when column searching is carried out, longitudinal verification is carried out on table data in the column;

when longitudinal verification is carried out, all the disclosure items of the columns are directly added, and if the total number of all the columns is balanced with the total number, the inspection is directly passed; finding the father and son items as long as the total number and the total number of one column are unbalanced; if the parent and child items are not found, finding the parent-child relationship through standard codes, wherein the standard codes are assigned when the head of the row list is standardized; when all columns meet the balance between the child item summation and the parent item, only taking the child item to participate in statistics, and otherwise, taking the parent item to participate in statistics; after determining which parent and child items need to participate, the entire table passes the vertical check when all columns satisfy the balance between the total value of the items participating in the calculation and the total item.

8. The method of constructing a logical structure tree of a listed company report as claimed in claim 5 wherein: when column searching is carried out, the table data in the column is transversely checked;

when the transverse verification is carried out, items which can be determined to be added and subtracted are calculated, and other items are calculated in a trial mode; if the project trial calculation has symbols, judging whether the current period is increased or decreased by using the symbols of trial calculation; if the trial calculation is unsigned, the first six bits are used for judging whether the current period is increased or decreased, and when the first six bits cannot be judged, the first six bits are used for judging whether the current period is increased or decreased.

9. The method of constructing a logical structure tree of a listed company report as claimed in claim 5 wherein: when column searching is carried out, deformation is carried out on a table in the column; the step of transforming the table in the column comprises the following steps:

searching the table according to the html tag;

merging the cross-page forms at the file level;

performing table deformation at a file level;

extracting the information of the position, the context and the unit currency of each table;

and generating a table object, wherein the table found under the column is allocated with the column ID, and the tables of the blocks outside the column are allocated with default column IDs.

10. A system for constructing a logical structure tree of a listed company report, comprising: the system comprises a conversion module, a preprocessing module, an extraction module and an extraction module;

the conversion module is used for converting the listed company report into a html format document;

the preprocessing module is used for preprocessing the html format document to obtain an html hierarchical structure document;

the extraction module is used for acquiring a preset structure of a logic structure tree and extracting nodes of each level of the preset structure from the html hierarchical structure document;

the extraction module is used for extracting matching contents based on the nodes of all levels and in a text matching mode to obtain the logic structure tree.

11. The listing company reported logical structure tree building system of claim 10, wherein: the conversion module converting the listing company report into the html format document includes the steps of:

acquiring a listed company report in a PDF format;

12. The listing company reported logical structure tree building system of claim 10, wherein: the preprocessing module is used for preprocessing the html format document, and the step of acquiring the html hierarchical structure document comprises the following steps:

identifying a directory of the html-format document;

and deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

13. The listing company reported logical structure tree building system of claim 10, wherein: the extraction module obtains a preset structure of a logic structure tree, and the extraction of each level of nodes of the preset structure in the html hierarchical structure document comprises the following steps:

identifying a hierarchical structure of the html hierarchically structured document;

based on the hierarchical structure, recursively searching all directly subordinate child nodes of the preset structure by adopting a strategy of from top to bottom, and from the top to the bottom;

after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes;

and analogizing in turn until the next-level node cannot be searched in the coverage area.

14. The listing company reported logical structure tree building system of claim 10, wherein: the extraction module extracts matching contents based on the nodes at each level and in a text matching mode to obtain the logic structure tree, and the extraction module comprises the following steps:

verifying the logic structure tree based on the html document directory;

15. The listing company reported logical structure tree building system of claim 14, wherein: when column searching is carried out, identifying a table in the column; identifying the table in the column includes the steps of:

extracting the header of the table and carrying out alias standardization;

ordering the list head after the alias standardization according to the first letter to generate a character string;

converting the character string into MD5 value, and performing table identification based on the MD5 value.

16. The listing company reported logical structure tree building system of claim 14, wherein: when column searching is carried out, longitudinal verification is carried out on table data in the column;

17. The listing company reported logical structure tree building system of claim 14, wherein: when column searching is carried out, the table data in the column is transversely checked;

18. The listing company reported logical structure tree building system of claim 14, wherein: when column searching is carried out, deformation is carried out on a table in the column; the step of transforming the table in the column comprises the following steps:

searching the table according to the html tag;

merging the cross-page forms at the file level;

performing table deformation at a file level;

extracting the information of the position, the context and the unit currency of each table;

and generating a table object, wherein the table found under the column is allocated with the column ID, and the tables of the blocks outside the column are allocated with default column IDs.

19. A storage medium on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method of constructing a logical structure tree of a company-to-market report of any one of claims 1 to 9.

20. A logical structure tree building terminal for a listed company report, comprising: a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory to cause the logical structure tree construction terminal of the public company report to perform the logical structure tree construction method of the public company report according to any one of claims 1 to 9.

Technical Field

The invention relates to the technical field of data processing, in particular to a method, a system, a storage medium and a terminal for constructing a logical structure tree reported by a listed company.

Background

The reporting of the listed company is a report that the board of the listed company regularly issues to stockholders or shareholders, and is a comprehensive report about the activities in the previous reporting period. The contents of a general quarterly newspaper, semiannual newspaper, or yearbook (hereinafter, collectively referred to as a standard newspaper) must be written according to rules specified by a trade on the market.

The newspaper is usually distributed on the internet in the form of an electronic document. Document content typically has a fixed directory style and hierarchical structure. The hierarchical structure has great research value and application value for understanding of both human beings and machines.

However, in the prior art, there is no efficient and accurate method for extracting the hierarchical structure of the newspaper electronic document, and the document content cannot be extracted into the tree-shaped hierarchical structure accurately and in a customized manner.

Disclosure of Invention

In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a method, a system, a storage medium, and a terminal for constructing a logical structure tree of a public company report, wherein specific node information is extracted to construct the logical structure tree based on a hierarchical structure of the public company report, thereby realizing effective processing of the public company report information.

In order to achieve the above objects and other related objects, the present invention provides a method for constructing a logical structure tree reported by a listed company, comprising the steps of: converting the listed company report into a html format document; preprocessing the html-format document to obtain an html hierarchical structure document; acquiring a preset structure of a logic structure tree, and extracting nodes of each level of the preset structure from the html hierarchical structure document; and extracting matching contents based on the nodes of each level in a text matching mode to obtain the logic structure tree.

In one embodiment of the present invention, converting the listed company report into html format document includes the following steps:

acquiring a listed company report in a PDF format;

In an embodiment of the present invention, the preprocessing the html format document, and obtaining the html hierarchical structure document includes the following steps:

identifying a directory of the html-format document;

and deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

In an embodiment of the present invention, obtaining a preset structure of a logical structure tree, and extracting nodes at each level of the preset structure from the html hierarchical structure document includes the following steps:

identifying a hierarchical structure of the html hierarchically structured document;

based on the hierarchical structure, recursively searching all directly subordinate child nodes of the preset structure by adopting a strategy of from top to bottom, and from the top to the bottom;

after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes;

and analogizing in turn until the next-level node cannot be searched in the coverage area.

In an embodiment of the present invention, extracting matching contents based on the nodes at each level in a text matching manner to obtain the logical structure tree includes the following steps:

verifying the logic structure tree based on the html document directory;

In an embodiment of the present invention, when column searching is performed, the column searching includes identifying a table in the column; identifying the table in the column includes the steps of:

extracting the header of the table and carrying out alias standardization;

ordering the list head after the alias standardization according to the first letter to generate a character string;

converting the character string into MD5 value, and performing table identification based on the MD5 value.

In an embodiment of the present invention, when column searching is performed, longitudinal verification is performed on table data in the column;

In an embodiment of the present invention, when column searching is performed, horizontal verification is performed on table data in the column;

In an embodiment of the present invention, when column searching is performed, the method includes transforming a table in the column; the step of transforming the table in the column comprises the following steps:

searching the table according to the html tag;

merging the cross-page forms at the file level;

performing table deformation at a file level;

extracting the information of the position, the context and the unit currency of each table;

and generating a table object, wherein the table found under the column is allocated with the column ID, and the tables of the blocks outside the column are allocated with default column IDs.

The invention provides a logic structure tree construction system for a company-on-market report, which comprises a conversion module, a preprocessing module, an extraction module and an extraction module, wherein the conversion module is used for converting a logic structure tree into a logic structure tree;

the conversion module is used for converting the listed company report into a html format document;

the preprocessing module is used for preprocessing the html format document to obtain an html hierarchical structure document;

the extraction module is used for acquiring a preset structure of a logic structure tree and extracting nodes of each level of the preset structure from the html hierarchical structure document;

the extraction module is used for extracting matching contents based on the nodes of all levels and in a text matching mode to obtain the logic structure tree.

In an embodiment of the present invention, the converting module converts the listed company report into the html format document, including the following steps:

acquiring a listed company report in a PDF format;

In an embodiment of the present invention, the preprocessing module preprocesses the html format document, and acquiring the html hierarchical structure document includes the following steps:

identifying a directory of the html-format document;

and deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

In an embodiment of the present invention, the extracting module obtains a preset structure of a logical structure tree, and extracting nodes at each level of the preset structure from the html hierarchical structure document includes the following steps:

identifying a hierarchical structure of the html hierarchically structured document;

based on the hierarchical structure, recursively searching all directly subordinate child nodes of the preset structure by adopting a strategy of from top to bottom, and from the top to the bottom;

after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes;

and analogizing in turn until the next-level node cannot be searched in the coverage area.

In an embodiment of the present invention, the extracting module extracts matching contents based on the nodes at each level in a text matching manner to obtain the logical structure tree, including the following steps:

verifying the logic structure tree based on the html document directory;

extracting the header of the table and carrying out alias standardization;

ordering the list head after the alias standardization according to the first letter to generate a character string;

converting the character string into MD5 value, and performing table identification based on the MD5 value.

In an embodiment of the present invention, when column searching is performed, longitudinal verification is performed on table data in the column;

In an embodiment of the present invention, when column searching is performed, horizontal verification is performed on table data in the column;

searching the table according to the html tag;

merging the cross-page forms at the file level;

performing table deformation at a file level;

extracting the information of the position, the context and the unit currency of each table;

and generating a table object, wherein the table found under the column is allocated with the column ID, and the tables of the blocks outside the column are allocated with default column IDs.

The present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described logical structure tree construction method for a marketing company report.

The invention provides a logical structure tree construction terminal reported by listed companies, which comprises: a processor and a memory;

the memory is used for storing a computer program;

the processor is used for executing the computer program stored in the memory so as to enable the logical structure tree construction terminal reported by the listed companies to execute the logical structure tree construction method reported by the listed companies.

As described above, the method, system, storage medium, and terminal for constructing a logical structure tree reported by a listed company according to the present invention have the following advantageous effects:

(1) extracting specific node information to construct a logical structure tree based on the hierarchical structure reported by the listed companies;

(2) the information can be verified in an automatic verification mode, so that the verification efficiency of the company report on the market is effectively improved;

(3) the time for business personnel to manually standardize the report of the listed company can be reduced, and the efficiency for standardizing the report structuralization of the listed company is improved;

(4) the method is beneficial to the acquisition and processing of the information reported by the listed companies, and has high practicability.

Drawings

FIG. 1 is a flow diagram illustrating a method for constructing a logical structure tree for a listed company report in accordance with one embodiment of the present invention;

FIG. 2 is a flow chart illustrating node extraction for a logical structure tree according to one embodiment of the present invention;

FIG. 3 is a flow chart illustrating table extraction according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a system for constructing a logical structure tree for a listed company report according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a terminal for constructing a logical structure tree reported by a listed company according to an embodiment of the invention.

Description of the element reference numerals

41 conversion module

42 preprocessing module

43 extraction module

44 extraction module

51 processing unit

52 memory

521 random access memory

522 cache memory

523 storage system

524 program/utility

5241 program module

53 bus

54I/O interface

55 network adapter

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.

It should be noted that the drawings provided in the present embodiment are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

The method, the system, the storage medium and the terminal for constructing the logical structure tree reported by the listed companies analyze the hierarchical structure of the reported by the listed companies and extract the specific node information according to the preset hierarchical structure of the logical structure tree, thereby realizing the construction of the logical structure tree, being beneficial to the effective processing and application of the reported information of the listed companies and having great practicability.

As shown in fig. 1, in an embodiment, the method for constructing a logical structure tree reported by listed companies of the present invention includes the following steps:

and step S1, converting the listed company report into an html format document.

Specifically, the company-on-market report needs to be converted into recognizable information first to facilitate subsequent information extraction.

In one embodiment of the present invention, converting the listed company report into html format document includes the following steps:

11) a PDF formatted listing company report is obtained.

Where the listed company reports are typically in PDF format. The PDF format can encapsulate characters, formats, colors, graphic images independent of equipment and resolution and the like in a file, can also contain electronic information such as hypertext links, sounds, dynamic images and the like, supports a very long file, has high integration level and safety and reliability, and meets the application requirements reported by listed companies.

12) And converting the PDF-format listed company report into an html-format document, wherein the html-format document retains all text contents and format information of the PDF-format listed company report.

Specifically, the PDF format listed company report is converted into a hypertext markup language (html) format document based on a PDF conversion function. The document format can be unified through the tags, and words and diagrams in the text are associated with other information media through a hyperlink method. The information resources distributed at different positions are connected in a random mode by the information organization mode, and convenience is provided for searching and retrieving information. In the invention, the html-format document retains all text contents and format information reported by listed companies in PDF format, thereby facilitating subsequent information processing.

And step S2, preprocessing the html format document to obtain an html hierarchical structure document.

Specifically, in order to facilitate extraction of hierarchical structure information in a company-on-market report, directory information therein needs to be first culled. Therefore, in an embodiment of the present invention, the step of preprocessing the html-format document to obtain the html-hierarchy document includes the following steps:

21) identifying a catalog of the html-formatted document.

And analyzing the html-format document and identifying the directory content in the html-format document.

22) And deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

Specifically, a regular matching algorithm is adopted to delete the directory text information in the html-format document, so that an html-hierarchy document only containing a hierarchy structure is obtained. Wherein each level is recorded in the form of a marker.

And step S3, acquiring a preset structure of the logic structure tree, and extracting nodes of each level of the preset structure from the html hierarchical structure document.

Specifically, a preset structure of a logical structure tree reported by a required listed company is preset for different requirements. Different announcement types and different enterprise types have different preset structures. The preset structure is set based on historical disclosure conditions of various announcements and published information disclosure standards. Before setting a preset structure of a certain announcement type, the historical original titles (from the first level to the minimum level) of the announcements are analyzed, the setting is started from the first level titles according to text clustering and the guidance of expert opinions and public disclosure, and then the subordinate second level titles are set based on the standard first level titles, and the process is circulated until the setting of the minimum level is completed. In order to ensure the reasonableness and effectiveness of the preset structure setting, coverage rate and multiplexing rate (for evaluation) are used, wherein the detailed flow of extracting each level of nodes of the preset structure is shown in fig. 2.

And searching matched nodes at all levels in the html hierarchical structure file according to the preset structure and the mark of the html hierarchical structure document.

31) identifying a hierarchy of the html hierarchy document.

Specifically, according to the mark information of the html hierarchical structure document, the corresponding hierarchical structure of the html hierarchical structure document is identified. Among them, the hierarchical structure has various forms.

Form one

First section

(A)

1、

(1)

Form two

First section

1、

(1)

32) Based on the hierarchical structure, all the directly subordinate child nodes of the preset structure are recursively searched by adopting a strategy of from top to bottom, and from the top to the bottom, and then from the top to the bottom.

Specifically, according to the preset structure, the directly subordinate child nodes are searched from top to bottom and from outside to inside.

33) And after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes.

Specifically, a next-level node, namely a directly subordinate grandchild node, is searched in the coverage range of each directly subordinate child node in a progressive manner.

34) And analogizing in turn until the next-level node cannot be searched in the coverage area.

Specifically, the step-by-step progression indicates that the node extraction of the logical structure tree is completed when the next-level node cannot be found in the coverage range of the newly found node.

And step S4, extracting matching contents based on the nodes of each level in a text matching mode to obtain the logic structure tree.

Specifically, after the extraction of each level of nodes of the logical structure tree is completed, text contents corresponding to each level of nodes need to be extracted to construct the logical structure tree.

In an embodiment of the present invention, extracting matching contents based on the nodes at each level in a text matching manner to obtain the logical structure tree includes the following steps:

41) and checking the logic structure tree based on the directory of the html format document.

Specifically, first, the nodes of the logical structure tree need to be checked according to the directory of the html-format document. Wherein, the title in the directory needs to be standardized according to the alias of the first-level tree, and the standardized first-level tree is required to be present in the standardized directory.

42) After the verification is passed, column searching is carried out; and matching corresponding column objects on the subtrees of the logic structure tree, wherein the column objects inherit from the subtree objects until the column searching of the logic structure tree is finished.

Specifically, if the verification is passed, column searching is carried out; if the verification fails, the process is ended, and the logical structure tree construction fails. Wherein the column refers to a range block of an analysis target. A column has one or more sub-columns. And matching corresponding column objects on the subtrees of the logic structure tree when column searching is carried out. The column object inherits from the sub-tree object, and its own attributes include the starting and ending position, unit, currency, etc. of the column. After a specific column is specified, information such as tables, texts and pictures under the column can be extracted.

After the column is searched, a table needs to be searched in the middle of the column. Each form generates a form object, the form object inherits from a column object, the unique identifier is a form serial number, and the attributes of the form object comprise form context characters, units in characters above the form, currency, dom objects and the like.

In an embodiment of the present invention, the table processing includes the following modes:

(1) form morphing

As shown in fig. 3, when performing table transformation, transforming the table at the file level includes performing merge cell and split cell processing on the table, merging header information, merging cross-page tables, splitting a spliced table, and the like. And for the column blocks and the blocks except the identified columns under the indexes, searching the table according to the html marks, merging the cross-page tables at the file level, performing table deformation at the file level, extracting the information such as the position, the context, the unit currency and the like of each table, finally generating a table object, distributing the column ID to the tables found under the columns, and distributing the default column ID to the tables of the blocks except the columns. Wherein the basis for performing the merging judgment comprises; (1) the characters between adjacent tables only have page numbers; (2) the number of columns of adjacent tables is the same.

(2) Form standardization

When the form is standardized, preliminary standardization is carried out on form element column item information and item information, the standardization mainly comprises the standardization of dates (YYYY/MM/DD, MM month DD day in YYYY, and the like), prefix and suffix cleaning (notes, comments, and the like), unit standardization (thousand yuan, thousand yuan RMB, thousand, and the like) and the standardization of other specific expressions, and the preliminary conversion function is to realize the standardization of a form header and the extraction of field attribute information hidden in the form.

(3) Table object extraction

In order to facilitate conversion, the dom objects of the table are converted into a two-dimensional array InnerTableItem in a memory, so that subsequent standardization and data extraction are facilitated. InnerTableItem is a cell object. The meanings of the various attributes of InnerTableItem are as follows:

TABLE 1 Attribute meanings of InnerTableItem

(4) Form recognition

Not only the elements in the table need to be extracted, but also the elements in the table need to be standardized (for example, item standardization, header standardization, etc.), and the table needs to be identified before the table is standardized (different target tables call different standard configurations).

The table identification method mainly used in the invention is a fingerprint identification method, extracts the table head of the table, standardizes the alias, generates a character string by ordering the standardized table head elements according to the initial letters, and converts the character string into an MD5 value, which is the fingerprint of the table. And identifying the form type according to the form fingerprint, and adopting different form standardization strategies.

(5) Table hierarchical column normalization

If the column entry parameter has multiple levels, each level is normalized separately, wherein the column entry normalization refers to a self-set standard parameter.

1) Standard parameter setting

Specifically, information such as actually disclosed parameter names and parameter corresponding numerical values of listed companies is extracted; creating standard parameters based on the actual disclosure condition of a listed company by combining text clustering and expert suggestions; managing the alias of the standard parameter based on the numerical trial calculation model and the text similarity; and evaluating the availability of the standard parameters from the dimensions of the coverage rate and the reuse rate based on the statistical model, and giving guidance for further maintenance.

2) Bulletin form parameter normalization

Specifically, based on the set alias of the standard parameter and the hierarchical relationship of the parent and child parameters, the parameter is standardized by combining the actual disclosure value of the parameter.

(6) Automatic check of table data and automatic supplement and correction of data

Specifically, based on data longitudinal level trial calculation and transverse formula trial calculation, the data sheet disclosure accuracy and the program standardization accuracy are verified and announced.

Specifically, the longitudinal check logic is as follows:

(a) all column disclosures are summed directly, if the total number of all columns balances the sum, then pass the test directly;

(b) if the total number of a column is unbalanced with the total number, finding out the parent and child items through the 'middle' mark;

(c) if the parent and child items are not found through the method, the parent and child relations are searched through the standard codes, and the standard codes are assigned when the head of the row and column is standardized. Storing the items with parent-child relationship as a tree structure, traversing all the first-level item trees, directly marking a mark participating in calculation if the items have no child nodes, if the current item has child nodes, balancing all the column parent-child items, and then participating in calculation by the child nodes, otherwise, participating in calculation by the parent nodes, and if the layer number of the first-level item trees is more than 2 levels, pruning from the deepest subtree.

(d) And when all the columns meet the balance between the child item summation and the parent item, only taking the child item to participate in statistics, and otherwise, taking the parent item to participate in statistics.

(e) After determining which parent and child items need to participate, the entire table passes the vertical check when all columns satisfy the balance between the total value of the items participating in the calculation and the total item.

It should be noted that if there is an unknown item, the unknown item performs trial calculation, that is, it is determined whether the sum of the unknown item and other items is balanced with the total value. If the trial calculation can balance, the current column is balanced. If the unknown item exists, firstly, whether the value of the unknown item in the current column is empty is checked; if the current item is empty and the computation tag of the current item is empty, the current item does not participate in computation, otherwise, the current item is listed in a trial computation list. If the trial calculation can be balanced, the current column is balanced, otherwise it is unbalanced.

The transverse check logic is as follows: firstly, calculating items which can be determined to be added or subtracted to obtain a value, and performing trial calculation on the other items; if the project trial calculation has symbols, judging whether the project trial calculation is increased or decreased in the current period by using the symbols of the trial calculation; if the trial calculation is unsigned, the first six bits are used for judging whether the sign is increased or decreased, and when the first six bits cannot be judged, the sign is judged by using the positive number or the negative number of the first six bits. Specifically, when there are other items in addition to the four items in the lateral direction, the other items are tentatively calculated.

Meanwhile, unknown or missing data are automatically supplemented and corrected based on a set formula.

As shown in FIG. 4, in one embodiment, the system for constructing a logical structure tree of a listed company report of the present invention comprises a transformation module 41, a preprocessing module 42, an extraction module 43, and an extraction module 44.

The conversion module 41 is used for converting the listed company report into html format document;

specifically, the company-on-market report needs to be converted into recognizable information first to facilitate subsequent information extraction.

In one embodiment of the present invention, converting the listed company report into html format document includes the following steps:

11) a PDF formatted listing company report is obtained.

The preprocessing module 42 is connected to the conversion module 41, and is configured to preprocess the html format document and obtain an html hierarchical structure document.

21) identifying a catalog of the html-formatted document.

And analyzing the html-format document and identifying the directory content in the html-format document.

22) And deleting the directory text information in the html-format document based on a regular matching algorithm, and obtaining the html hierarchical structure document.

The extraction module 43 is connected to the preprocessing module 42, and is configured to obtain a preset structure of the logical structure tree, and extract nodes at different levels of the preset structure from the html hierarchical structure document.

And searching matched nodes at all levels in the html hierarchical structure file according to the preset structure and the mark of the html hierarchical structure document.

31) identifying a hierarchy of the html hierarchy document.

Form one

First section

(A)

1、

(1)

Form two

First section

1、

(1)

Specifically, according to the preset structure, the directly subordinate child nodes are searched from top to bottom and from outside to inside.

33) And after all the direct child nodes are searched, searching and extracting all the direct grandchild nodes according to the coverage range of the direct child nodes.

Specifically, a next-level node, namely a directly subordinate grandchild node, is searched in the coverage range of each directly subordinate child node in a progressive manner.

34) And analogizing in turn until the next-level node cannot be searched in the coverage area.

The extracting module 44 is connected to the converting module 41 and the extracting module 42, and is configured to extract matching contents based on the nodes at each level and in a text matching manner to obtain the logical structure tree.

In an embodiment of the present invention, extracting matching contents based on the nodes at each level in a text matching manner to obtain the logical structure tree includes the following steps:

41) and checking the logic structure tree based on the directory of the html format document.

In an embodiment of the present invention, the table processing includes the following modes:

(1) form morphing

(2) Form standardization

(3) Table object extraction

(4) Form recognition

(5) Table hierarchical column normalization

If the column entry parameter has multiple levels, each level is normalized separately, wherein the column entry normalization refers to a self-set standard parameter.

1) Standard parameter setting

2) Bulletin form parameter normalization

(6) Automatic check of table data and automatic supplement and correction of data

Specifically, the longitudinal check logic is as follows:

(a) all column disclosures are summed directly, if the total number of all columns balances the sum, then pass the test directly;

(b) if the total number of a column is unbalanced with the total number, finding out the parent and child items through the 'middle' mark;

Meanwhile, unknown or missing data are automatically supplemented and corrected based on a set formula.

It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the x module may be a processing element that is set up separately, or may be implemented by being integrated in a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and the function of the x module may be called and executed by a processing element of the apparatus. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

The storage medium of the present invention stores thereon a computer program that, when executed by a processor, implements the above-described logical structure tree construction method reported by a public company. The storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

Any combination of one or more storage media may be employed. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

In one embodiment, the logical structure tree construction terminal reported by listed companies of the present invention comprises: a processor and a memory.

The memory is for storing a computer program.

The memory includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

The processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the logic structure tree construction terminal reported by the listed company to execute the travel mode reminding method.

Preferably, the Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.

As shown in FIG. 5, the logical structure tree building terminal reported by the public companies of the present invention is represented in the form of a general purpose computing device. Components of the logical structure tree building terminal reported by the listed companies may include, but are not limited to: one or more processors or processing units 51, a memory 52, and a bus 53 that couples the various system components (including the memory 52 and the processing unit 51).

Bus 53 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

The logical structure tree building terminal reported by the public company typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the logical structure tree building terminals reported by the public company and include both volatile and nonvolatile media, removable and non-removable media.

The memory 52 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)521 and/or cache memory 522. The logical structure tree building terminal reported by the public company may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 523 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5 and commonly referred to as a "hard disk drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 53 by one or more data media interfaces. Memory 52 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in the memory 52, such program modules 5241 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may include an implementation of a network environment. The program modules 5241 generally perform the functions and/or methods of the described embodiments of the invention.

The logical structure tree building terminal reported by the public company may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the logical structure tree building terminal reported by the public company, and/or any device (e.g., network card, modem, etc.) that enables the logical structure tree building terminal reported by the public company to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 54. Also, the terminal 4 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 55. As shown in fig. 5, the network adapter 55 communicates via bus 53 with the other modules of the logical structure tree building terminal reported by the public company. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the logical structure tree building terminal reported by the listed companies, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In summary, the method, the system, the storage medium and the terminal for constructing the logical structure tree reported by the listed companies extract specific node information to construct the logical structure tree based on the hierarchical structure reported by the listed companies; the information can be verified in an automatic verification mode, so that the verification efficiency of the company report on the market is effectively improved; the time for business personnel to manually standardize the report of the listed company can be reduced, and the efficiency for standardizing the report structuralization of the listed company is improved; the method is beneficial to the acquisition and processing of the information reported by the listed companies, and has high practicability. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

28页详细技术资料下载

Method, system, storage medium and terminal for constructing logical structure tree reported by listed companies

相关技术

网友询问留言