阅读说明：本技术 一种搜题的方法、装置、电子设备和存储介质 (Method and device for searching questions, electronic equipment and storage medium ) 是由 何华强 于 2020-06-29 设计创作，主要内容包括：本发明实施例公开了一种搜题的方法、装置、电子设备和存储介质。该方法包括：在电子设备处于指读场景下,接收拍照指令,利用图像采集装置对承载体件拍照；对初始图像进行识别,得到初始图像中每一个题号对应的内部轮廓；根据题号的数值和级别设置每个内部轮廓对应的标签；接收并识别用户发出的第一语音指令中的目标题号；根据目标题号确定目标标签以及目标内部轮廓；根据目标内部轮廓确定目标图像；对目标图像进行OCR识别,并利用识别的结果在数据库中搜索匹配的试题。实施本发明实施例,可以完全避免手指在指读场景下对试题内容遮挡造成的识别内容损失,提高裁取试图片的识别内容完整性以及推送用户试题原题率,提高用户学习的交互体验。(The embodiment of the invention discloses a method and a device for searching questions, electronic equipment and a storage medium. The method comprises the following steps: receiving a photographing instruction when the electronic equipment is in a finger reading scene, and photographing the carrier piece by using an image acquisition device; identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image; setting a label corresponding to each internal contour according to the numerical value and the grade of the question number; receiving and identifying a target question number in a first voice command sent by a user; determining a target label and a target internal contour according to the target question number; determining a target image according to the internal contour of the target; and performing OCR recognition on the target image, and searching the database for a matched test question by using a recognition result. By implementing the embodiment of the invention, the loss of the identification content caused by the shielding of the test question content by fingers in a finger reading scene can be completely avoided, the integrity of the identification content of the cut test picture and the test question rate of the user can be improved, and the interactive experience of the user in learning can be improved.)

1. A method for searching for a question, comprising:

receiving a photographing instruction sent by a user when the electronic equipment is in a finger reading scene, and photographing the carrier piece by using an image acquisition device to obtain an initial image;

identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image;

setting a label corresponding to each internal contour according to the numerical value and the grade of the question number;

receiving and identifying a target question number in a first voice command sent by a user;

determining a target label and a target internal contour according to the target question number, wherein the target label is a label matched with the target question number, and the target internal contour is an internal contour associated with the target label;

determining a text outline according to the target internal outline, and segmenting an initial image in the text outline to obtain a target image;

and performing OCR recognition on the target image, and searching a database for matched test questions by using the recognition result.

2. The method of claim 1, wherein identifying the initial image to obtain an inner contour corresponding to each topic number in the initial image comprises:

inputting the initial image into a question recognition network model, a text line detection network model and a question number detection network model based on deep learning in parallel to determine a question outline, a text line outline and a question number frame;

creating a blank mask image, wherein the blank mask image has the same size as the initial image;

adding the title contour to the mask map;

determining the upper boundary of an item number line according to the item number frame and the text line outline, and adding the upper boundary into the mask image;

and extending the left end point and the right end point of the upper boundary so as to connect the upper boundary with the theme contour, wherein the theme contour is divided into a plurality of theme areas by the upper boundary, and each theme area forms an internal contour corresponding to each theme number.

3. The method of claim 1, wherein setting the label corresponding to each internal contour according to the value and level of the question mark comprises:

obtaining the grade of each question number through a question number classification model, wherein the grade comprises a primary question and a secondary question;

and setting a label for the question mark according to the numerical value of the question mark and the grade of the question mark, wherein the label embodies the value corresponding to the question mark and the grade of the question mark.

4. The method of claim 3, wherein determining a target label and a target inner contour according to the target question number, the target label being a label matched with the target question number, the target inner contour being an inner contour associated with the target label, comprises:

traversing all the tags according to the target question number, and determining the tags matched with the target question number as target tags;

when the question number corresponding to the target label is a secondary question and the number of the target label is only one, taking the inner contour corresponding to the target label as the target inner contour;

when the question number corresponding to the target label is a first-level question, or/and the target labels are multiple, or no target label exists, sending an interaction instruction to a user;

and receiving a second voice instruction sent by the user according to the interaction instruction, and determining a new target question number according to the second voice instruction until the question number corresponding to the determined target label is a secondary question and the number of the target label is only one.

5. The method according to any one of claims 1-4, wherein receiving and identifying the target topic number in the first voice command issued by the user comprises:

receiving a first voice instruction sent by a user, and extracting one or more digital keywords in the first voice instruction, or one or more digital keywords and associated words of the digital keywords;

and taking the information corresponding to the digital key words or the relevant words of the digital key words and the digital key words as target question numbers.

6. The method according to any one of claims 1 to 4, wherein determining a text contour from the target internal contour and segmenting an initial image within the text contour to obtain a target image comprises:

and taking the target internal contour as a text contour, and segmenting the initial image to obtain a target image, wherein the target image is an initial image part in the text contour.

7. The method of any one of claims 1-4, wherein performing OCR recognition on the target image and using the recognition results to search a database for matching questions, comprises:

performing OCR recognition on the target image to obtain a recognition result;

searching in a database to obtain target test questions, wherein the similarity between the target test questions and the recognition result is greater than or equal to a preset threshold value;

and if the similarity between the test questions in the database and the recognition result is smaller than a preset threshold, selecting a preset number of test questions with the highest similarity with the recognition result as target test questions.

8. An apparatus for searching for a topic, the apparatus comprising:

the photographing unit is used for receiving a photographing instruction sent by a user when the electronic equipment is in a finger reading scene, and photographing the carrier piece by using the image acquisition device to obtain an initial image;

the identification unit is used for identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image;

the setting unit is used for setting a label corresponding to each internal contour according to the numerical value and the grade of the question number;

the receiving unit is used for receiving and identifying a target question number in a first voice command sent by a user;

the determining unit is used for determining a target label and a target internal contour according to the target question mark, wherein the target label is a label matched with the target question mark, and the target internal contour is an internal contour associated with the target label;

the segmentation unit is used for determining a text outline according to the target internal outline and segmenting an initial image in the text outline to obtain a target image;

and the searching unit is used for performing OCR recognition on the target image and searching the matched test questions in a database by using the recognition result.

9. The apparatus of claim 8, wherein the identification unit comprises:

the input subunit is used for inputting the initial image into a question identification network model, a text line detection network model and a question number detection network model based on deep learning in parallel to determine a question outline, a text line outline and a question number frame;

a creating subunit, configured to create a blank mask map, where the blank mask map has the same size as the initial image;

an adding subunit, configured to add the title contour to the mask map;

the boundary determining subunit is used for determining the upper boundary of the question mark line according to the question mark frame and the text line outline, and adding the upper boundary into the mask image;

and the extension subunit is used for extending the left end point and the right end point of the upper boundary so as to connect the upper boundary with the theme contour, the theme contour is divided into a plurality of theme areas by the upper boundary, and each theme area forms an internal contour corresponding to each theme number.

10. The apparatus of claim 8, wherein the setting unit comprises:

the classification subunit is used for acquiring the grade of each question number through the question number classification model, wherein the grade comprises a primary question and a secondary question;

and the label setting subunit is used for setting a label for the question number according to the numerical value of the question number and the grade of the question number, and the label embodies the value corresponding to the question number and the grade of the question number.

11. The apparatus of claim 10, wherein the determining unit comprises:

the traversal subunit is used for traversing all the tags according to the target question number, and determining the tags matched with the target question number as target tags;

the judging subunit is configured to, when the question number corresponding to the target tag is a secondary question and only one target tag is available, take the internal contour corresponding to the target tag as a target internal contour;

the feedback subunit is used for sending an interaction instruction to a user when the question number corresponding to the target label is a first-level question, or/and the target labels are multiple, or the target labels do not exist; and receiving a second voice instruction sent by the user according to the interaction instruction, and determining a new target question number according to the second voice instruction until the question number corresponding to the determined target label is a secondary question and the number of the target label is only one.

12. The apparatus according to any one of claims 8-11, wherein the receiving unit comprises:

the extraction subunit is used for receiving a first voice instruction sent by a user and extracting one or more digital keywords in the first voice instruction, or one or more digital keywords and associated words of the digital keywords;

and the target question number determining subunit is used for taking the information corresponding to the digital key words or the relevant words of the digital key words as target question numbers.

13. The apparatus according to any one of claims 8-11, wherein the search unit comprises:

the OCR recognition subunit is used for performing OCR recognition on the target image to obtain a recognition result;

the calculation subunit is used for searching in the database to obtain target test questions, and the similarity between the target test questions and the recognition results is greater than or equal to a preset threshold value;

and the pushing subunit is used for selecting the preset number of test questions with the highest similarity to the identification result as the target test questions when the similarities of the test questions in the database and the identification result are smaller than a preset threshold value.

14. An electronic device, comprising: a memory storing executable program code; a processor coupled with the memory; the processor calls the executable program code stored in the memory for executing a method of searching for topics as claimed in any one of claims 1 to 7.

15. A computer-readable storage medium storing a computer program, wherein the computer program causes a computer to perform a method of searching for a subject according to any one of claims 1 to 7.

Technical Field

The invention relates to the technical field of intelligent terminals, in particular to a method and a device for searching questions, electronic equipment and a storage medium.

Background

Many present electronic teaching auxiliary equipment have mostly and indicate the scene of reading, and current indicate read the scene and indicate that the user points to when supporting bodies such as books, exercise book or examination paper through the finger, teaching auxiliary equipment can shoot the supporting body through image acquisition device to the position of discernment finger, thereby confirm the user's intention according to the finger position, and then obtain the image that the user's intention corresponds, be used for the original question search etc.. At present, the shot images of the bearing body are all pictures when the fingers of the user read, and the fingers (palm, pen holding and the like) can inevitably shield a part of effective test question data, so that part of the data is lost, the matching degree of OCR recognition and test question searching according to the recognition content is influenced, and the test question rate finally pushed to the user is low.

Disclosure of Invention

Aiming at the defects, the embodiment of the invention discloses a method and a device for searching questions, electronic equipment and a storage medium, which can avoid finger from blocking images of a bearer and improve the rate of pushing test questions of a user.

The first aspect of the embodiments of the present invention discloses a method for searching questions, where the method includes:

receiving a photographing instruction sent by a user when the electronic equipment is in a finger reading scene, and photographing the carrier piece by using an image acquisition device to obtain an initial image;

identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image;

setting a label corresponding to each internal contour according to the numerical value and the grade of the question number;

receiving and identifying a target question number in a first voice command sent by a user;

determining a target label and a target internal contour according to the target question number, wherein the target label is a label matched with the target question number, and the target internal contour is an internal contour associated with the target label;

determining a text outline according to the target internal outline, and segmenting an initial image in the text outline to obtain a target image;

and performing OCR recognition on the target image, and searching a database for matched test questions by using the recognition result.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, identifying the initial image to obtain an internal contour corresponding to each topic number in the initial image includes:

inputting the initial image into a question recognition network model, a text line detection network model and a question number detection network model based on deep learning in parallel to determine a question outline, a text line outline and a question number frame;

creating a blank mask image, wherein the blank mask image has the same size as the initial image;

adding the title contour to the mask map;

determining the upper boundary of an item number line according to the item number frame and the text line outline, and adding the upper boundary into the mask image;

and extending the left end point and the right end point of the upper boundary so as to connect the upper boundary with the theme contour, wherein the theme contour is divided into a plurality of theme areas by the upper boundary, and each theme area forms an internal contour corresponding to each theme number.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, setting a label corresponding to each internal contour according to the value and the level of the title number includes:

obtaining the grade of each question number through a question number classification model, wherein the grade comprises a primary question and a secondary question;

and setting a label for the question mark according to the numerical value of the question mark and the grade of the question mark, wherein the label embodies the value corresponding to the question mark and the grade of the question mark.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining a target label and a target inner contour according to the target question mark, where the target label is a label adapted to the target question mark, and the target inner contour is an inner contour associated with the target label, includes:

traversing all the tags according to the target question number, and determining the tags matched with the target question number as target tags;

when the question number corresponding to the target label is a secondary question and the number of the target label is only one, taking the inner contour corresponding to the target label as the target inner contour;

when the question number corresponding to the target label is a first-level question, or/and the target labels are multiple, or no target label exists, sending an interaction instruction to a user;

and receiving a second voice instruction sent by the user according to the interaction instruction, and determining a new target question number according to the second voice instruction until the question number corresponding to the determined target label is a secondary question and the number of the target label is only one.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, the receiving and recognizing a target question number in a first voice instruction issued by a user includes:

receiving a first voice instruction sent by a user, and extracting one or more digital keywords in the first voice instruction, or one or more digital keywords and associated words of the digital keywords;

and taking the information corresponding to the digital key words or the relevant words of the digital key words and the digital key words as target question numbers.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining a text contour according to the target internal contour, and segmenting an initial image in the text contour to obtain a target image, includes:

and taking the target internal contour as a text contour, and segmenting the initial image to obtain a target image, wherein the target image is an initial image part in the text contour.

As an alternative implementation manner, in the first aspect of the embodiment of the present invention, performing OCR recognition on the target image, and searching a database for a matching test question using a result of the recognition includes:

performing OCR recognition on the target image to obtain a recognition result;

searching in a database to obtain target test questions, wherein the similarity between the target test questions and the recognition result is greater than or equal to a preset threshold value;

and if the similarity between the test questions in the database and the recognition result is smaller than a preset threshold, selecting a preset number of test questions with the highest similarity with the recognition result as target test questions.

A second aspect of the embodiments of the present invention discloses a device for searching questions, including:

the photographing unit is used for receiving a photographing instruction sent by a user when the electronic equipment is in a finger reading scene, and photographing the carrier piece by using the image acquisition device to obtain an initial image;

the identification unit is used for identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image;

the setting unit is used for setting a label corresponding to each internal contour according to the numerical value and the grade of the question number;

the receiving unit is used for receiving and identifying a target question number in a first voice command sent by a user;

the determining unit is used for determining a target label and a target internal contour according to the target question mark, wherein the target label is a label matched with the target question mark, and the target internal contour is an internal contour associated with the target label;

the segmentation unit is used for determining a text outline according to the target internal outline and segmenting an initial image in the text outline to obtain a target image;

and the searching unit is used for performing OCR recognition on the target image and searching the matched test questions in a database by using the recognition result.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the identification unit includes:

the input subunit is used for inputting the initial image into a question identification network model, a text line detection network model and a question number detection network model based on deep learning in parallel to determine a question outline, a text line outline and a question number frame;

a creating subunit, configured to create a blank mask map, where the blank mask map has the same size as the initial image;

an adding subunit, configured to add the title contour to the mask map;

the boundary determining subunit is used for determining the upper boundary of the question mark line according to the question mark frame and the text line outline, and adding the upper boundary into the mask image;

and the extension subunit is used for extending the left end point and the right end point of the upper boundary so as to connect the upper boundary with the theme contour, the theme contour is divided into a plurality of theme areas by the upper boundary, and each theme area forms an internal contour corresponding to each theme number.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the setting unit includes:

the classification subunit is used for acquiring the grade of each question number through the question number classification model, wherein the grade comprises a primary question and a secondary question;

and the label setting subunit is used for setting a label for the question number according to the numerical value of the question number and the grade of the question number, and the label embodies the value corresponding to the question number and the grade of the question number.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the determining unit includes:

the traversal subunit is used for traversing all the tags according to the target question number, and determining the tags matched with the target question number as target tags;

the judging subunit is configured to, when the question number corresponding to the target tag is a secondary question and only one target tag is available, take the internal contour corresponding to the target tag as a target internal contour;

the feedback subunit is used for sending an interaction instruction to a user when the question number corresponding to the target label is a first-level question, or/and the target labels are multiple, or the target labels do not exist; and receiving a second voice instruction sent by the user according to the interaction instruction, and determining a new target question number according to the second voice instruction until the question number corresponding to the determined target label is a secondary question and the number of the target label is only one.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the receiving unit includes:

the extraction subunit is used for receiving a first voice instruction sent by a user and extracting one or more digital keywords in the first voice instruction, or one or more digital keywords and associated words of the digital keywords;

and the target question number determining subunit is used for taking the information corresponding to the digital key words or the relevant words of the digital key words as target question numbers.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the search unit includes:

the OCR recognition subunit is used for performing OCR recognition on the target image to obtain a recognition result;

the calculation subunit is used for searching in the database to obtain target test questions, and the similarity between the target test questions and the recognition results is greater than or equal to a preset threshold value;

and the pushing subunit is used for selecting the preset number of test questions with the highest similarity to the identification result as the target test questions when the similarities of the test questions in the database and the identification result are smaller than a preset threshold value.

A third aspect of an embodiment of the present invention discloses an electronic device, including: a memory storing executable program code; a processor coupled with the memory; the processor calls the executable program code stored in the memory to execute part or all of the steps of the method for searching the topic disclosed by the first aspect of the embodiment of the invention.

A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, where the computer program enables a computer to execute part or all of the steps of the method for searching for a topic disclosed in the first aspect of the present invention.

A fifth aspect of the embodiments of the present invention discloses a computer program product, which, when running on a computer, causes the computer to execute part or all of the steps of the method for searching for a topic disclosed in the first aspect of the embodiments of the present invention.

A sixth aspect of the present invention discloses an application publishing platform, where the application publishing platform is configured to publish a computer program product, and when the computer program product runs on a computer, the computer is enabled to execute some or all of the steps of the method for searching for a topic disclosed in the first aspect of the present invention.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, when the electronic equipment is in a finger reading scene, a photographing instruction sent by a user is received, and an image acquisition device is used for photographing the carrier part to obtain an initial image; identifying the initial image to obtain an internal contour corresponding to each question mark in the initial image; setting a label corresponding to each internal contour according to the numerical value and the grade of the question number; receiving and identifying a target question number in a first voice command sent by a user; determining a target label and a target internal contour according to the target question number, wherein the target label is a label matched with the target question number, and the target internal contour is an internal contour associated with the target label; determining a text outline according to the target internal outline, and segmenting an initial image in the text outline to obtain a target image; and performing OCR recognition on the target image, and searching a database for matched test questions by using the recognition result. Therefore, by implementing the embodiment of the invention, the loss of the identification content caused by the shielding of the test question content by fingers (palm, pen holding and the like) in a finger reading scene can be completely avoided, so that the integrity of the identification content of the cut test picture is improved, the test question rate of the pushed user is improved, the intention of the user is met to the maximum extent, and the interactive experience of the user in learning is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow chart illustrating a method for searching for a topic according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a preview state of an image capturing device according to an embodiment of the disclosure;

FIG. 3 is a schematic diagram of an internal contour obtaining method according to an embodiment of the disclosure;

FIG. 4 is a schematic diagram of an initial image according to an embodiment of the present invention;

FIG. 5 is a schematic view of a title profile disclosed in an embodiment of the present invention;

FIG. 6 is a schematic illustration of an internal profile disclosed in an embodiment of the present invention;

FIG. 7 is a schematic diagram of a carrier according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a page structure of another carrier according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a page structure of another carrier according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of an apparatus for searching for a question according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second", "third", "fourth", and the like in the description and the claims of the present invention are used for distinguishing different objects, and are not used for describing a specific order. The terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a method, a device, electronic equipment and a storage medium for searching questions, which can completely avoid the loss of identification contents caused by the shielding of test question contents by fingers (palm, pen holding and the like) in a finger reading scene, thereby improving the integrity of the identification contents of a cut test picture, further improving the rate of pushing test questions of a user, meeting the intention of the user to the maximum extent and improving the interactive experience of the learning of the user, and are described in detail below by combining with the attached drawings.