阅读说明：本技术 基于图像识别的数据解读方法、装置、设备及存储介质 (Data interpretation method, device and equipment based on image recognition and storage medium ) 是由 张俊杰 于 2021-08-30 设计创作，主要内容包括：本发明涉及人工智能领域,公开了一种基于图像识别的数据解读方法、装置、设备及存储介质,用于解决现有技术中对不同版式内容的体检报告进行解读时效率低下的问题。该方法包括：接收并提取数据解读请求中的体检报告图片；识别其中的体检结果的文本位置,并进行裁剪得到裁剪图像集；对裁剪图像集中的裁剪图像块进行文本识别,得到文本内容；并确定裁剪图像块中包含的检查类别以及检查结果,并获取对应的标准指标范围；判断检查结果是否在标准指标范围内；若否,则将检查结果和检查类别标注为异常检查结果和异常检查类别；在医学知识图谱中检索出医疗信息,并输出数据解读结果。此外,本发明还涉及区块链技术,体检报告的相关信息可存储于区块链中。(The invention relates to the field of artificial intelligence, and discloses a data interpretation method, a data interpretation device, data interpretation equipment and a storage medium based on image recognition, which are used for solving the problem of low efficiency in interpretation of physical examination reports with different format contents in the prior art. The method comprises the following steps: receiving and extracting a physical examination report picture in the data reading request; recognizing the text position of the physical examination result, and cutting to obtain a cut image set; performing text recognition on the cut image blocks in the cut image set to obtain text contents; determining the inspection type and the inspection result contained in the cut image block, and acquiring a corresponding standard index range; judging whether the inspection result is in the standard index range; if not, marking the checking result and the checking type as an abnormal checking result and an abnormal checking type; and retrieving medical information in the medical knowledge map, and outputting a data interpretation result. In addition, the invention also relates to the block chain technology, and the related information of the physical examination report can be stored in the block chain.)

1. A data interpretation method based on image recognition, characterized in that the physical examination report interpretation method comprises:

receiving a data reading request, and extracting physical examination report pictures contained in the data reading request;

calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cutting the physical examination report picture according to the text position to obtain a cut image set, wherein the cut image set comprises at least one cut image block;

calling a preset optical character recognition model to perform text recognition on the cut image blocks to obtain text contents in the cut image blocks;

determining inspection types contained in the cut image blocks and inspection results contained in the inspection types according to the text content, and acquiring corresponding standard index ranges according to the inspection types, wherein the number of the inspection results is at least one;

judging whether the checking result is in the standard index range or not;

if not, marking the check result as an abnormal check result, and marking the check type as an abnormal check type;

and retrieving in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

2. The image-recognition-based data interpretation method according to claim 1, wherein the determining, from the text content, the inspection category included in the cropped image block and the inspection result included in the inspection category, and the obtaining the corresponding standard indicator range according to the inspection category includes:

calling a preset named entity model to identify named entities contained in the text content to obtain detection items, and extracting a detection result corresponding to the detection items from the text content;

according to the detection items, finding out the detection categories corresponding to the detection items in a preset medical knowledge map;

and extracting standard index ranges corresponding to all detection items contained in the examination categories from the medical knowledge graph according to the examination categories.

3. The image recognition-based data interpretation method according to claim 1, wherein the finding of the examination category corresponding to the detection item in a preset medical knowledge map according to the detection item comprises:

calling a preset classification judgment model to judge the alternative inspection category to which the detection item belongs according to the detection item;

finding out standard examination items contained in the alternative examination categories in a preset medical knowledge map;

judging whether the detection item is the same as the standard check item;

and if so, taking the alternative checking type as the checking type corresponding to the detection item.

4. The image-recognition-based data interpretation method according to claim 3, wherein the receiving a data interpretation request and extracting a physical examination report picture included in the data interpretation request includes:

receiving a data interpretation request, and extracting a PDF format file of a physical examination report contained in the data interpretation request;

analyzing the PDF format file to obtain a binary file of the physical examination report;

and converting the binary file into a picture format file through a preset conversion tool to obtain a physical examination report picture, wherein the preset conversion tool is constructed according to a PDF2image tool.

5. The image recognition-based data interpretation method according to claim 4, further comprising, after the converting the binary file into a picture format file by a preset conversion tool:

calling a preset direction adjusting tool to identify the direction of the picture format file to obtain an actual direction angle of a picture, and rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a uniform direction angle;

and detecting whether the first picture file has tensile deformation, if so, calling a preset perspective adjustment tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file.

6. The image recognition-based data interpretation method of claim 5, wherein the calling a preset content positioning model to recognize a text position of the physical examination result in the physical examination report picture, and cropping the physical examination report picture according to the text position to obtain a cropped image set comprises:

inputting the physical examination report picture into a preset content positioning model to identify the text content range of the content in the physical examination report picture, so as to obtain the boundary point coordinates of the text position of the physical examination result;

cutting the physical examination report picture according to the boundary point coordinates to obtain at least one text image block;

and scaling at least one text image block in an equal ratio to obtain at least one cut image block, and forming a cut image set based on the at least one cut image block, wherein the length of the short side of the cut image block is a preset length.

7. The image recognition-based data interpretation method according to any one of claims 1 to 6, further comprising, after the outputting of the data interpretation result according to the medical information:

calculating a difference index of the abnormal checking result in the abnormal checking category and the standard index range;

inquiring the risk diseases and the risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index;

and inquiring corresponding medical suggestions according to the risk diseases and the risk probability, and outputting the medical suggestions.

8. An image recognition-based data interpretation apparatus, characterized by comprising:

the extraction module is used for receiving a data reading request and extracting a physical examination report picture contained in the data reading request;

an image cutting module, configured to call a preset content positioning model to identify a text position of a physical examination result in the physical examination report picture, and cut the physical examination report picture according to the text position to obtain a cut image set, where the cut image set includes at least one cut image block

The text recognition module is used for calling a preset optical character recognition model to perform text recognition on the cut image blocks to obtain text contents in the cut image blocks;

the content determining module is used for determining the checking type contained in the cutting image block and the checking result contained in the checking type according to the text content and acquiring the corresponding standard index range according to the checking type;

the judging module is used for judging whether the checking result is in the standard index range or not;

if not, the marking module is used for marking the inspection result as an abnormal inspection result and marking the inspection type as an abnormal inspection type;

and the output module is used for retrieving in a preset medical knowledge map according to the abnormal examination result to obtain the medical information corresponding to the abnormal examination category and outputting a data interpretation result according to the medical information.

9. An image recognition-based data interpretation apparatus characterized by comprising: a memory and at least one processor, the memory having instructions stored therein;

the at least one processor invokes the instructions in the memory to cause the image recognition based data interpretation apparatus to perform the steps of the image recognition based data interpretation method of any one of claims 1-7.

10. A computer readable storage medium having instructions stored thereon, which when executed by a processor implement the steps of the image recognition based data interpretation method according to any one of claims 1 to 7.

Technical Field

The invention relates to the field of artificial intelligence, in particular to a data interpretation method, a data interpretation device, data interpretation equipment and a storage medium based on image recognition.

Background

With the development of social economy, the living standard of people is improved, and the basic medical guarantee system of urban and rural residents is continuously improved, so that the demand of people on medical services, particularly the demand on health management such as physical examination and the like, is increasing. In order to meet the physical examination requirements of different social groups, hospitals continuously improve the physical examination operation process of medical care personnel in the reformation and development, provide a humanized physical examination management mode, and increase various functions of supporting disease auxiliary diagnosis, health management, remote consultation and the like.

In the prior art, the method for automatically reading the physical examination reports can only identify the contents in the physical examination reports of the same format, however, the types of the physical examination reports of different physical examination organizations or different examination items are different, and the identification model needs to be retrained for each layout, so that the reading method of the physical examination reports is difficult to be commonly used, and the reading efficiency is further low.

Disclosure of Invention

The invention mainly aims to solve the problem of low efficiency in interpretation of physical examination reports with different format contents in the prior art.

The invention provides a data interpretation method based on image recognition, which comprises the following steps: receiving a data reading request, and extracting physical examination report pictures contained in the data reading request; calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cutting the physical examination report picture according to the text position to obtain a cut image set, wherein the cut image set comprises at least one cut image block; calling a preset optical character recognition model to perform text recognition on the cut image blocks to obtain text contents in the cut image blocks; determining inspection types contained in the cut image blocks and inspection results contained in the inspection types according to the text content, and acquiring corresponding standard index ranges according to the inspection types, wherein the number of the inspection results is at least one; judging whether the checking result is in the standard index range or not; if not, marking the check result as an abnormal check result, and marking the check type as an abnormal check type; and retrieving in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

Optionally, in a first implementation manner of the first aspect of the present invention, the determining, according to the text content, an inspection category included in the cropped image block and an inspection result included in the inspection category, and acquiring a corresponding standard indicator range according to the inspection category includes: calling a preset named entity model to identify named entities contained in the text content to obtain detection items, and extracting a detection result corresponding to the detection items from the text content; according to the detection items, finding out the detection categories corresponding to the detection items in a preset medical knowledge map; and extracting standard index ranges corresponding to all detection items contained in the examination categories from the medical knowledge graph according to the examination categories.

Optionally, in a second implementation manner of the first aspect of the present invention, the finding, according to the detection item, an examination category corresponding to the detection item in a preset medical knowledge graph includes: calling a preset classification judgment model to judge the alternative inspection category to which the detection item belongs according to the detection item; finding out standard examination items contained in the alternative examination categories in a preset medical knowledge map; judging whether the detection item is the same as the standard check item; and if so, taking the alternative checking type as the checking type corresponding to the detection item.

Optionally, in a third implementation manner of the first aspect of the present invention, the receiving a data interpretation request and extracting a physical examination report picture included in the data interpretation request includes: receiving a data interpretation request, and extracting a PDF format file of a physical examination report contained in the data interpretation request; analyzing the PDF format file to obtain a binary file of the physical examination report; and converting the binary file into a picture format file through a preset conversion tool to obtain a physical examination report picture, wherein the preset conversion tool is constructed according to a PDF2image tool.

Optionally, in a fourth implementation manner of the first aspect of the present invention, after the converting the binary file into the picture format file by a preset conversion tool, the method further includes: calling a preset direction adjusting tool to identify the direction of the picture format file to obtain an actual direction angle of a picture, and rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a uniform direction angle; and detecting whether the first picture file has tensile deformation, if so, calling a preset perspective adjustment tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file.

Optionally, in a fifth implementation manner of the first aspect of the present invention, the calling a preset content positioning model to identify a text position of a physical examination result in the physical examination report picture, and cutting the physical examination report picture according to the text position to obtain a cut image set includes: inputting the physical examination report picture into a preset content positioning model to identify the text content range of the content in the physical examination report picture, so as to obtain the boundary point coordinates of the text position of the physical examination result; cutting the physical examination report picture according to the boundary point coordinates to obtain at least one text image block; and scaling at least one text image block in an equal ratio to obtain at least one cut image block, and forming a cut image set based on the at least one cut image block, wherein the length of the short side of the cut image block is a preset length.

Optionally, in a sixth implementation manner of the first aspect of the present invention, after the outputting a data interpretation result according to the medical information, the method further includes: calculating a difference index of the abnormal checking result in the abnormal checking category and the standard index range; inquiring the risk diseases and the risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index; and inquiring corresponding medical suggestions according to the risk diseases and the risk probability, and outputting the medical suggestions.

A second aspect of the present invention provides a data interpretation apparatus based on image recognition, including: the extraction module is used for receiving a data reading request and extracting a physical examination report picture contained in the data reading request; the image cutting module is used for calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture and cutting the physical examination report picture according to the text position to obtain a cut image set, wherein the cut image set comprises at least one cut image block text identification module and is used for calling a preset optical character identification model to perform text identification on the cut image block to obtain the text content in the cut image block; the content determining module is used for determining the checking type contained in the cutting image block and the checking result contained in the checking type according to the text content and acquiring the corresponding standard index range according to the checking type; the judging module is used for judging whether the checking result is in the standard index range or not; if not, the marking module is used for marking the inspection result as an abnormal inspection result and marking the inspection type as an abnormal inspection type; and the output module is used for retrieving in a preset medical knowledge map according to the abnormal examination result to obtain the medical information corresponding to the abnormal examination category and outputting a data interpretation result according to the medical information.

Optionally, in a first implementation manner of the second aspect of the present invention, the content determining module includes: the content identification unit is used for calling a preset named entity model to identify the named entity contained in the text content to obtain a detection item, and extracting a check result corresponding to the detection item from the text content; the category searching unit is used for searching out the checking category corresponding to the detection item in a preset medical knowledge map according to the detection item; and the range extraction unit is used for extracting a standard index range corresponding to each detection item contained in the examination category from the medical knowledge graph according to the examination category.

Optionally, in a second implementation manner of the second aspect of the present invention, the category searching unit includes: the category judgment subunit is used for calling a preset classification judgment model to judge the alternative inspection category to which the detection item belongs according to the detection item; the standard item searching subunit is used for searching the standard examination items contained in the alternative examination categories in a preset medical knowledge map; an item judgment subunit, configured to judge whether the detection item is the same as the standard inspection item; and the inspection type determining subunit is configured to, if yes, use the candidate inspection type as the inspection type corresponding to the detection item.

Optionally, in a third implementation manner of the second aspect of the present invention, the extraction module includes: the file extraction unit is used for receiving a data reading request and extracting a PDF format file of a physical examination report contained in the data reading request; the file analysis unit is used for analyzing the PDF format file to obtain a binary file of the physical examination report; and the picture conversion unit is used for converting the binary file into a picture format file through a preset conversion tool to obtain a physical examination report picture, wherein the preset conversion tool is constructed according to a PDF2image tool.

Optionally, in a fourth implementation manner of the second aspect of the present invention, the extraction module further includes: the direction conversion unit is used for calling a preset direction adjustment tool to perform direction identification on the picture format file to obtain an actual direction angle of a picture, and rotating the picture format file according to the actual direction angle to obtain a first adjustment picture file with a uniform direction angle; and the deformation adjusting unit is used for detecting whether the first picture file has tensile deformation or not, if so, calling a preset perspective adjusting tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file.

Optionally, in a fifth implementation manner of the second aspect of the present invention, the image cropping module includes: the content position determining unit is used for inputting the physical examination report picture into a preset content positioning model to identify the text content range of the content in the physical examination report picture so as to obtain the boundary point coordinates of the text position of the physical examination result; the cutting unit is used for cutting the physical examination report picture according to the boundary point coordinates to obtain at least one text image block; and the scaling unit is used for scaling at least one text image block in an equal ratio to obtain at least one cut image block, and forming a cut image set based on the at least one cut image block, wherein the length of the short side of the cut image block is a preset length.

Optionally, in a sixth implementation manner of the second aspect of the present invention, the data interpretation apparatus based on image recognition further includes a medical advice generation module, where the medical advice generation module includes: a difference index calculation unit configured to calculate a difference index between the abnormality check result in the abnormality check category and the standard index range; the query unit is used for querying the risk diseases and the risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index; and the suggestion output unit is used for inquiring the corresponding medical suggestion according to the risk diseases and the risk probability and outputting the medical suggestion.

A third aspect of the present invention provides a data interpretation apparatus based on image recognition, including: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the image recognition based data interpretation apparatus to perform the steps of the image recognition based data interpretation method described above.

A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the above-described data interpretation method based on image recognition.

The technical scheme provided by the invention can be applied to disease auxiliary diagnosis and treatment and remote consultation, and the method receives the data interpretation request and extracts the physical examination report picture contained in the data interpretation request; calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cutting the physical examination report picture according to the text position to obtain a cut image set, wherein the cut image set comprises at least one cut image block; calling a preset optical character recognition model to perform text recognition on the cut image blocks to obtain text contents; determining the checking type contained in the cutting image block and the checking result contained in the checking type according to the text content, and acquiring a corresponding standard index range according to the checking type, wherein the number of the checking results is at least one; judging whether the inspection result is in the standard index range; if not, marking the checking result and the checking type as an abnormal checking result and an abnormal checking type; and retrieving in a preset medical knowledge map according to the abnormal inspection result to obtain corresponding medical information, and outputting a data interpretation result according to the medical information. According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, so that the interpretation efficiency of the physical examination reports is improved.

Drawings

Fig. 1 is a schematic diagram of a first embodiment of a data interpretation method based on image recognition in the embodiment of the invention;

FIG. 2 is a diagram illustrating a second embodiment of a data interpretation method based on image recognition according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a data interpretation method based on image recognition according to a third embodiment of the present invention;

fig. 4 is a schematic diagram of a fourth embodiment of a data interpretation method based on image recognition in the embodiment of the invention;

FIG. 5 is a schematic diagram of an embodiment of a data interpretation device based on image recognition according to the embodiment of the invention;

fig. 6 is a schematic diagram of another embodiment of the data interpretation device based on image recognition in the embodiment of the invention;

fig. 7 is a schematic diagram of an embodiment of a data interpretation device based on image recognition in the embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a data interpretation method, a device, equipment and a storage medium based on image identification, which are used for receiving and extracting a physical examination report picture contained in a data interpretation request; recognizing the text position of the physical examination result in the picture, and cutting to obtain a cut image set; performing text recognition on the cut image blocks in the cut image set to obtain text contents; determining the inspection type and the inspection result contained in the cut image block, and acquiring a corresponding standard index range; judging whether the inspection result is in the standard index range; if not, marking the checking result and the checking type as an abnormal checking result and an abnormal checking type; and retrieving medical information in the medical knowledge map, and outputting a data interpretation result. According to the scheme in the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, so that the interpretation efficiency of the physical examination reports is improved; the method can be applied to disease auxiliary diagnosis and treatment and remote consultation.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For easy understanding, a detailed flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a data interpretation method based on image recognition according to an embodiment of the present invention includes:

101. receiving a data reading request, and extracting physical examination report pictures contained in the data reading request;

it is to be understood that the executing main body of the present invention may be a physical examination report interpretation device, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.

The physical examination report interpretation method in the embodiment can identify the physical examination report to be interpreted by the user and output the interpreted information, so that the user can quickly know the physical examination conclusion and the physical examination information in the physical examination report.

The server in this embodiment first receives a data interpretation request from a user, where the data interpretation request may be a physical examination report interpretation request, the report interpretation request includes a physical examination report file to be interpreted, and after receiving the report interpretation request, extracts the physical examination report file included in the report interpretation request and converts the obtained physical examination report file into a physical examination report picture; specifically, the physical examination report picture in the embodiment may be a report picture obtained by scanning a physical examination report file or a physical examination report picture obtained by shooting; the medical examination report file can also be a PDF format file, and the medical examination report file in the PDF format can be converted into a picture format through a preset format conversion tool to obtain a medical examination report picture and perform subsequent identification. In addition, the physical examination report file may be a plurality of pages, and when the format conversion is performed in the case where the physical examination report file is a plurality of pages, the converted pictures are arranged in the order of the original page number.

102. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture;

103. cutting the physical examination report picture according to the text position to obtain a cut image set;

inputting the obtained physical examination report picture into a preset content positioning model to identify the text position contained in the physical examination report picture, wherein the content positioning model is established by a DBNet (differential binary network) tool.

In this embodiment, the format of the physical examination report pictures of different physical examination items and physical examination units may be greatly different, for example, some pictures may contain tables of laboratory test reports and test results, some pictures may contain video reports, and the different reports have different word positions; when the physical examination mechanisms are different, the backgrounds of the physical examination report pictures may be different, the accuracy in text recognition is reduced due to the differences, the training time of the recognition model is increased by establishing different recognition models for different formats, and therefore the text position of the physical examination result contained in the physical examination report picture is recognized by adopting the content positioning model in the step.

After the position of the text content in each physical examination report picture is identified, the text content is cut according to the text position of the text content, the content which does not contain the text content such as images, template patterns and the like and is irrelevant to the physical examination result is removed, cut image blocks are obtained, all the obtained cut image blocks form a cut image set, wherein the cut image set comprises at least one cut image block.

104. Calling a preset optical character recognition model to perform text recognition on the cut image block to obtain text contents in the cut image block;

and then, transmitting the cut image blocks in the obtained cut image set into a preset Optical Character Recognition (OCR) model to recognize text contents in the physical examination report.

Specifically, after receiving the clipped image block, the optical character recognition model in this step first performs tilt correction on the physical examination report picture, and if it is detected that the physical examination report picture has a tilt, detects a tilt angle, and performs perspective transformation on the picture according to the tilt angle to correct the tilt.

After the processing is finished, recognizing characters in the cut image blocks, recognizing the edge of each character in the character part, and cutting the cut image blocks according to the character edges to obtain character images. Then, each character image is recognized to obtain a recognition result of each character, and each character is converted into a language which can be recognized by a computer; and regularizing and correcting the conversion result to finally obtain the text content.

105. Determining the checking type contained in the cutting image block and the checking result contained in the checking type according to the text content;

106. acquiring a corresponding standard index range according to the inspection type;

in this embodiment, an unadjusted Named Entity Recognition (NER) tool is obtained in advance; simultaneously acquiring a medical dictionary, performing parameter adjustment on a named entity recognition tool which is not adjusted according to the content in the medical dictionary to obtain a named entity recognition model, calling the named entity recognition model to recognize the named entity on the obtained text content, and acquiring the specific meaning of the recognized named entity by combining the content in the medical dictionary so as to recognize the inspection category corresponding to each detection item contained in the text content; determining a checking result corresponding to the checking type according to the character distribution position in the physical examination report picture; and after the examination type is obtained, inquiring a standard index range corresponding to the examination type in a preset medical knowledge base.

107. Judging whether the inspection result is in the standard index range;

108. if not, marking the inspection result as an abnormal inspection result, and marking the inspection type as an abnormal inspection type;

109. and searching in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

And after the standard index range corresponding to the inspection type is obtained, judging whether all the inspection results in the inspection type are abnormal or not.

Specifically, it is determined whether each inspection result exceeds its corresponding standard index range. If all the inspection results do not exceed the standard index range corresponding to the inspection item, the inspection type is considered to be in a normal state, and a preset notification text that the inspection type is normal is output, wherein the preset notification text that the inspection type is normal may further include a health life guide input in advance, for example: balanced diet, moderate exercise, etc.

And if at least one detection item exceeding the standard index range exists in the detection result, marking the detection result as an abnormal detection result, and marking the detection type as an abnormal detection type.

According to the abnormal examination category, medical information corresponding to each abnormal examination result is retrieved in a preset medical knowledge map according to each abnormal examination result, wherein the medical information can be the reason which possibly causes the abnormality.

According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, so that the interpretation efficiency of the physical examination reports is improved.

Referring to fig. 2, a second embodiment of the data interpretation method based on image recognition according to the embodiment of the present invention includes:

201. receiving a data reading request, and extracting physical examination report pictures contained in the data reading request;

202. calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture;

203. cutting the physical examination report picture according to the text position to obtain a cut image set;

204. calling a preset optical character recognition model to perform text recognition on the cut image block to obtain text contents in the cut image block;

in this embodiment, the contents in steps 201 to 204 are substantially the same as those in steps 101 to 104 in the previous embodiment, and therefore, the description thereof is omitted.

205. Calling a preset named entity model to identify named entities contained in the text content to obtain detection items, and extracting a detection result corresponding to the detection items from the text content;

206. calling a preset classification judgment model to judge the alternative inspection types of the detection items according to the detection items;

in this embodiment, parameter adjustment is performed on the non-adjusted NER tool in advance according to the content in the medical dictionary to obtain a named entity recognition model, and the named entity recognition model is called to perform recognition and labeling of the named entity in the text on the obtained text content to obtain a detection item in the text content.

After the detection items in the text content are obtained, the actual detection result corresponding to the detection category is determined according to the character distribution position in the physical examination report picture.

Calling a classification judgment model established in advance according to a CRF (Conditional Random field) tool, inputting the recognized detection items into the classification judgment model, and obtaining inspection category judgment results to which the detection items possibly belong and confidence probabilities corresponding to the inspection category judgment results, wherein the number of the inspection category judgment results obtained in the step is at least one, sorting the inspection category judgment results according to the obtained confidence probabilities to obtain a judgment result sequence, and extracting at least one inspection category judgment result from the extraction judgment result sequence; specifically, the top several ranks or top several percentages of inspection category judgment results with the highest category probability can be screened out from the judgment result sequence as the candidate inspection categories to which the detection items belong.

207. Searching standard examination items contained in the alternative examination categories in a preset medical knowledge map;

208. judging whether the detection items are the same as the standard inspection items or not;

209. if yes, taking the alternative checking type as the checking type corresponding to the detection item;

210. extracting standard index ranges corresponding to all detection items contained in the examination categories from the medical knowledge graph according to the examination categories;

acquiring a preset medical knowledge map, wherein the preset knowledge map is obtained by arranging in advance according to data in a medical dictionary and a medical knowledge base; and inquiring standard detection items corresponding to each candidate detection category according to the medical knowledge graph, judging whether the obtained detection items belong to the corresponding standard detection items, and if so, taking the candidate detection category with the highest coincidence degree of the detection items and the standard detection items as the detection category corresponding to the detection items.

211. Judging whether the inspection result is in the standard index range;

212. if not, marking the inspection result as an abnormal inspection result, and marking the inspection type as an abnormal inspection type;

213. and searching in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

In this embodiment, the contents of steps 211 to 213 are substantially the same as those of steps 107 to 109 in the previous embodiment, and therefore are not described herein again.

The scheme in the embodiment of the invention can determine the category of the examination item to which the examination item belongs according to the content in the physical examination reports of different formats, determine the standard index range according to the examination category, automatically decode the content of the physical examination report according to the actual examination result and the marked index range to obtain the medical information in the physical examination report, and improve the accuracy and the interpretation efficiency of the physical examination report.

Referring to fig. 3, a third embodiment of the data interpretation method based on image recognition according to the embodiment of the present invention includes:

301. receiving a data reading request, and extracting a PDF format file of a physical examination report contained in the data reading request;

302. analyzing the PDF format file to obtain a binary file of the physical examination report;

303. converting the binary file into a picture format file through a preset conversion tool;

in this embodiment, a server first receives a report interpretation request from a user, where the report interpretation request includes a physical examination report file to be interpreted, and after receiving the report interpretation request, extracts a physical examination report file included in the report interpretation request, and converts the obtained physical examination report file into a picture format file.

And when the obtained picture format file is a plurality of pages, converting the PDF format file into a binary file according to the page number sequence, and inputting the obtained binary file into a preset picture conversion tool to obtain the picture format file.

304. Calling a preset direction adjusting tool to identify the direction of the picture format file to obtain the actual direction angle of the picture;

305. rotating the picture format file according to the actual direction angle to obtain a first adjustment picture file with a uniform direction angle;

after obtaining the converted picture format file, calling a preset direction adjustment tool to perform direction identification on the picture format file, where the direction adjustment tool in this step has a direction angle detection function and a rotation function, where the direction angle detection function may be constructed according to a deep neural network for feature extraction and a DFL classification network (DFL-CNN) for classifying features, where the deep neural network for feature extraction may be, for example, a VGG network (a deep neural network proposed by Visual Geometry Group) or a denset (dense connected convolutional network), and the like.

In the step, firstly, the characteristics of the picture format file are extracted, the directions of the file are classified according to the extracted characteristics to obtain an actual direction angle, and then the picture format file is rotated according to the actual direction angle to obtain a first adjustment picture file with a uniform direction angle.

306. Detecting whether the first picture file has stretching deformation;

307. if so, calling a preset perspective adjustment tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file to obtain a physical examination report picture;

and then, detecting whether the first picture file has stretching deformation or not, if so, performing inclination correction and perspective transformation operation on the physical examination report picture, and correcting the inclination and stretching deformation conditions to obtain a second adjustment picture file. And then, denoising the second adjustment picture file, so that the definition of the picture is improved, the accuracy of subsequent text recognition is improved, and a physical examination report picture is obtained.

308. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture;

309. cutting the physical examination report picture according to the text position to obtain a cut image set;

inputting the obtained physical examination report picture into a preset content positioning model to identify the text position contained in the physical examination report picture, wherein the content positioning model is established by a DBNet (differential binary network) tool.

In this embodiment, the formats of the physical examination report pictures of different physical examination items and physical examination units may be greatly different, the differences may reduce the accuracy in text recognition, and establishing different recognition models for each different format may increase the training time of the recognition models, so that the text position of the physical examination result included in the physical examination report picture is recognized by using the content positioning model in this step.

After the position of the text content in each physical examination report picture is identified, the text content is cut according to the text position of the text content, the content which does not contain the text content such as images, template patterns and the like and is irrelevant to the physical examination result is removed, cut image blocks are obtained, all the obtained cut image blocks form a cut image set, wherein the cut image set comprises at least one cut image block.

310. Calling a preset optical character recognition model to perform text recognition on the cut image block to obtain text contents in the cut image block;

and then, transmitting the cut image blocks in the obtained cut image set into a preset Optical Character Recognition (OCR) model to recognize text contents in the physical examination report.

Specifically, after receiving the clipped image block, the optical character recognition model in this step first performs tilt correction on the physical examination report picture, and if it is detected that the physical examination report picture has a tilt, detects a tilt angle, and performs perspective transformation on the picture according to the tilt angle to correct the tilt.

After the processing is finished, recognizing characters in the cut image blocks, recognizing the edge of each character in the character part, and cutting the cut image blocks according to the character edges to obtain character images. Then, each character image is recognized to obtain a recognition result of each character, and each character is converted into a language which can be recognized by a computer; and regularizing and correcting the conversion result to finally obtain the text content.

The optical character recognition and identification model can be constructed through a CRNN-CTC (connected Current Network-connected terminal Convolutional Classification, time Classification Convolutional Neural Network), character cutting is carried out on the images to be recognized in the image sets to be recognized through the recognition model, each character in the physical examination report pictures is recognized, each character is converted into a language which can be recognized by a computer, and text content is obtained.

In the embodiment, when text content is identified, DBNet and CRNN-CTC are combined to identify the text content, so that text paragraphs with different typesetting positions can be identified, and the universality of the physical examination report interpretation method in the embodiment can be improved, for example, the physical examination report interpretation method can accurately identify form type test sheet data, and can also accurately identify text paragraphs in image report types.

311. Determining the checking type contained in the cutting image block and the checking result contained in the checking type according to the text content, and acquiring a corresponding standard index range according to the checking type;

the method comprises the steps of constructing an NER model in advance based on BERT (bidirectional Encoder retrieval from transformations) and CRF (Conditional Random Fields) tools, wherein the NER model is obtained by training based on information in a medical dictionary and a medical knowledge base, classifying and judging current detection items based on the detection items, and calculating the probability of the detection categories to which the detection items possibly belong. And searching the examination category corresponding to the detection item and the standard index range corresponding to the examination category according to the obtained probability of the examination category and the information in the preset medical map.

312. Judging whether the inspection result is in the standard index range;

313. if not, marking the inspection result as an abnormal inspection result, and marking the inspection type as an abnormal inspection type;

314. and searching in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

The contents of steps 312 to 314 in this embodiment are substantially the same as those of steps 107 to 109 in the previous embodiment. And thus will not be described in detail herein.

According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, the interpretation efficiency of the physical examination reports is improved, and in addition, the accuracy of the information acquired by interpretation is also improved.

Referring to fig. 4, a fourth embodiment of the data interpretation method based on image recognition according to the embodiment of the present invention includes:

401. receiving a data reading request, and extracting a PDF format file of a physical examination report contained in the data reading request;

402. analyzing the PDF format file to obtain a binary file of the physical examination report;

403. converting the binary file into a picture format file through a preset conversion tool;

404. calling a preset direction adjusting tool to identify the direction of the picture format file to obtain the actual direction angle of the picture, and rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a uniform direction angle;

405. detecting whether the first picture file has stretching deformation;

406. if so, calling a preset perspective adjustment tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file to obtain a physical examination report picture;

the specific contents in steps 401 to 406 in this embodiment are substantially the same as those in steps 301 to 308 in the previous embodiment, and therefore, the detailed description thereof is omitted here.

407. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cutting the physical examination report picture according to the text position to obtain a cut image set;

and inputting the obtained physical examination report picture into a preset content positioning model to identify the text position contained in the physical examination report picture.

In this embodiment, the format of the physical examination report pictures of different physical examination items and physical examination units may be greatly different, for example, some pictures may contain tables of laboratory test reports and test results, some pictures may contain video reports, and the different reports have different word positions; when the physical examination mechanisms are different, the backgrounds of the physical examination report pictures may be different, the accuracy in text recognition is reduced due to the differences, the training time of the recognition model is increased by establishing different recognition models for different formats, and therefore the text position of the physical examination result contained in the physical examination report picture is recognized by adopting the content positioning model in the step.

After the position of the text content in each physical examination report picture is identified, the text content is cut according to the text position of the text content, the content which does not contain the text content such as images, template patterns and the like and is irrelevant to the physical examination result is removed, cut image blocks are obtained, all the obtained cut image blocks form a cut image set, wherein the cut image set comprises at least one cut image block.

Specifically, a content positioning model generated by a DBNet tool is called to detect the field position on the physical examination report picture, and boundary point coordinates of four points of the upper left point, the lower left point, the upper right point and the lower right point of the field are obtained after detection. And cutting to obtain at least one text image block based on the coordinates, then scaling the at least one text image block in an equal ratio, wherein the length of the short side of the cut image block is a preset length, in a specific example, the short side of the image block is scaled to 720 pixels, the long side is scaled in an equal ratio according to the size of the original image, and an image set to be recognized is formed based on the scaled images.

408. Calling a preset optical character recognition model to perform text recognition on the cut image block to obtain text contents in the cut image block;

409. calling a preset named entity model to identify named entities contained in the text content to obtain detection items, and extracting a detection result corresponding to the detection items from the text content;

410. calling a preset classification judgment model to judge the alternative inspection types of the detection items according to the detection items;

411. searching standard examination items contained in the alternative examination categories in a preset medical knowledge map;

412. judging whether the detection items are the same as the standard inspection items or not;

413. if yes, taking the alternative checking type as the checking type corresponding to the detection item;

414. extracting standard index ranges corresponding to all detection items contained in the examination categories from the medical knowledge graph according to the examination categories;

the contents of steps 408 to 414 in this embodiment are substantially the same as those of steps 206 to 210 in the previous embodiment, and therefore, the description thereof is omitted.

415. Judging whether the inspection result is in the standard index range;

416. if not, marking the inspection result as an abnormal inspection result, and marking the inspection type as an abnormal inspection type;

in this embodiment, the contents in steps 415 to 416 are substantially the same as those in steps 107 to 108 in the previous embodiment, and therefore, the description thereof is omitted.

417. And searching in a preset medical knowledge map according to the abnormal examination result to obtain medical information corresponding to the abnormal examination type, and outputting a data interpretation result according to the medical information.

And after the standard index range corresponding to the inspection type is obtained, judging whether all the inspection results in the inspection type are abnormal or not.

Specifically, it is determined whether each inspection result exceeds its corresponding standard index range. If all the inspection results do not exceed the standard index range corresponding to the inspection item, the inspection type is considered to be in a normal state, and a preset notification text that the inspection type is normal is output, wherein the preset notification text that the inspection type is normal may further include a health life guide input in advance, for example: balanced diet, moderate exercise, etc.

And if at least one detection item exceeding the standard index range exists in the detection result, marking the detection result as an abnormal detection result, and marking the detection type as an abnormal detection type.

According to the abnormal examination category, medical information corresponding to each abnormal examination result is retrieved in a preset medical knowledge map according to each abnormal examination result, wherein the medical information can be the reason which possibly causes the abnormality.

Further, calculating a difference index of an actual detection result of the abnormal item from the standard index; inquiring the risk diseases and the risk probability corresponding to the phase difference index in a medical knowledge graph according to the phase difference index; inquiring the risk diseases corresponding to the phase difference indexes in the medical knowledge graph through a clustering algorithm according to the medical suggestions corresponding to the risk diseases, wherein abnormal indexes (such as red cell count increase in blood routine) or abnormal conclusions (a breast color ultrasound report shows that left breast nodules exist) in each type of examination sheet can be detected by a DBScan algorithm in the clustering algorithm; and calculating the risk probability according to the specific content of the medical knowledge graph, and giving disease risk early warning and medical advice. In addition, in the application, all the generated medical suggestions can be counted, the specific contents of the medical suggestions can be analyzed, and the medical suggestions which conflict with each other can be modified to give overall medical suggestions.

According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, the interpretation efficiency of the physical examination reports is improved, and in addition, the accuracy of the information acquired by interpretation is also improved.

With reference to fig. 5, the data interpretation method based on image recognition according to the embodiment of the present invention is described above, and a data interpretation device based on image recognition according to the embodiment of the present invention is described below, where an embodiment of the data interpretation device based on image recognition according to the embodiment of the present invention includes:

an extracting module 501, configured to receive a data reading request, and extract a physical examination report picture included in the data reading request;

an image clipping module 502, configured to invoke a preset content positioning model to identify a text position of a physical examination result in the physical examination report picture, and clip the physical examination report picture according to the text position to obtain a clipped image set, where the clipped image set includes at least one clipped image block

The text recognition module 503 is configured to invoke a preset optical character recognition model to perform text recognition on the clipped image block, so as to obtain text content in the clipped image block;

a content determining module 504, configured to determine, according to the text content, an inspection category included in the cropped image block and an inspection result included in the inspection category, and obtain a corresponding standard indicator range according to the inspection category;

a judging module 505, configured to judge whether the inspection result is within the standard indicator range;

a labeling module 506, configured to label the inspection result as an abnormal inspection result and label the inspection type as an abnormal inspection type if the inspection result is not the abnormal inspection result;

and the output module 507 is used for retrieving in a preset medical knowledge graph according to the abnormal examination result to obtain medical information corresponding to the abnormal examination category, and outputting a data interpretation result according to the medical information.

According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, so that the interpretation efficiency of the physical examination reports is improved.

Referring to fig. 6, another embodiment of the data interpretation device based on image recognition according to the embodiment of the present invention includes:

an extracting module 501, configured to receive a data reading request, and extract a physical examination report picture included in the data reading request; the image cutting module 502 is configured to invoke a preset content positioning model to identify a text position of a physical examination result in the physical examination report picture, and cut the physical examination report picture according to the text position to obtain a cut image set, where the cut image set includes at least one cut image block text recognition module 503 and is configured to invoke a preset optical character recognition model to perform text recognition on the cut image block to obtain a text content in the cut image block; a content determining module 504, configured to determine, according to the text content, an inspection category included in the cropped image block and an inspection result included in the inspection category, and obtain a corresponding standard indicator range according to the inspection category; a judging module 505, configured to judge whether the inspection result is within the standard indicator range; a labeling module 506, configured to label the inspection result as an abnormal inspection result and label the inspection type as an abnormal inspection type if the inspection result is not the abnormal inspection result; and the output module 507 is used for retrieving in a preset medical knowledge graph according to the abnormal examination result to obtain medical information corresponding to the abnormal examination category, and outputting a data interpretation result according to the medical information.

In another embodiment of the present application, the content determining module 504 includes: a content identification unit 5041, configured to invoke a preset named entity model to identify a named entity included in the text content, to obtain a detection item, and extract a check result corresponding to the detection item from the text content; a category searching unit 5042, configured to search, according to the detection item, a checking category corresponding to the detection item in a preset medical knowledge graph; a range extraction unit 5043, configured to extract, from the medical knowledge graph according to the examination category, a standard index range corresponding to each detection item included in the examination category.

In another embodiment of the present application, the category lookup unit 5042 includes: the category judgment subunit is used for calling a preset classification judgment model to judge the alternative inspection category to which the detection item belongs according to the detection item; the standard item searching subunit is used for searching the standard examination items contained in the alternative examination categories in a preset medical knowledge map; an item judgment subunit, configured to judge whether the detection item is the same as the standard inspection item; and the inspection type determining subunit is configured to, if yes, use the candidate inspection type as the inspection type corresponding to the detection item.

In another embodiment of the present application, the extraction module 501 includes: a file extraction unit 5011 configured to receive a data reading request, and extract a PDF format file of a physical examination report included in the data reading request; the file analyzing unit 5012 is configured to analyze the PDF format file to obtain a binary file of the physical examination report; the picture conversion unit 5013 is configured to convert the binary file into a picture format file through a preset conversion tool, so as to obtain a physical examination report picture, where the preset conversion tool is constructed according to a PDF2image tool.

In another embodiment of the present application, the extracting module 501 further includes: the direction conversion unit is used for calling a preset direction adjustment tool to perform direction identification on the picture format file to obtain an actual direction angle of a picture, and rotating the picture format file according to the actual direction angle to obtain a first adjustment picture file with a uniform direction angle; and the deformation adjusting unit is used for detecting whether the first picture file has tensile deformation or not, if so, calling a preset perspective adjusting tool to perform perspective transformation on the rotating picture file to obtain a second adjusted picture file, and performing denoising processing on the second adjusted picture file.

In another embodiment of the present application, the image cropping module 502 comprises: a content position determining unit 5021, configured to input the physical examination report picture into a preset content positioning model to identify a text content range of the content in the physical examination report picture, so as to obtain a boundary point coordinate of a text position of a physical examination result; a cutting unit 5022, configured to cut the physical examination report picture according to the boundary point coordinates to obtain at least one text image block; the scaling unit 5023 is configured to perform equal-ratio scaling on at least one text image block to obtain at least one clipped image block, and form a clipped image set based on the at least one clipped image block, where a short edge length of the clipped image block is a preset length.

In another embodiment of the present application, the image recognition-based data interpretation apparatus further comprises a medical advice generation module comprising: a difference index calculation unit configured to calculate a difference index between the abnormality check result in the abnormality check category and the standard index range; the query unit is used for querying the risk diseases and the risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index; and the suggestion output unit is used for inquiring the corresponding medical suggestion according to the risk diseases and the risk probability and outputting the medical suggestion.

According to the scheme provided by the embodiment of the invention, the contents in the physical examination reports of different formats can be automatically interpreted to obtain the medical information in the physical examination reports, the interpretation efficiency of the physical examination reports is improved, and in addition, the accuracy of the information acquired by interpretation is also improved.

Fig. 5 and 6 describe the data interpretation device based on image recognition in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the data interpretation device based on image recognition in the embodiment of the present invention is described in detail from the perspective of hardware processing.

Fig. 7 is a schematic structural diagram of an image recognition-based data interpretation device 700 according to an embodiment of the present invention, which may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 710 (e.g., one or more processors) and a memory 720, one or more storage media 730 (e.g., one or more mass storage devices) storing an application 733 or data 732. Memory 720 and storage medium 730 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 730 may include one or more modules (not shown), each of which may include a series of instruction operations for the image recognition based data interpretation apparatus 700. Still further, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the image recognition based data interpretation device 700.

The image recognition based data interpretation device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input-output interfaces 760, and/or one or more operating systems 731, such as Windows server, Mac OS X, Unix, Linux, FreeBSD, and so forth. Those skilled in the art will appreciate that the configuration of the data interpretation apparatus based on image recognition shown in fig. 7 does not constitute a limitation of the data interpretation apparatus based on image recognition, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.

The present invention also provides a computer device, which may be any device capable of executing the data interpretation method based on image recognition described in the above embodiments, the computer device including a memory and a processor, the memory having stored therein computer readable instructions, when executed by the processor, causing the processor to execute the steps of the data interpretation method based on image recognition in the above embodiments.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

In one possible implementation, the data in the present invention is medical data, such as personal health records, prescriptions, examination reports, and the like.

In a possible implementation manner, the text content is a medical text, and the medical text may be a medical Electronic Record (Electronic Healthcare Record), an Electronic personal health Record, and a series of Electronic records with a stored value to be looked up, such as a medical Record, an electrocardiogram, and a medical image.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions that, when executed on a computer, cause the computer to perform the steps of the image recognition-based data interpretation method.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.