阅读说明：本技术 一种药品识别方法、装置和用于药品识别的装置 (Medicine identification method and device for medicine identification ) 是由 张婷婷 于 2019-08-15 设计创作，主要内容包括：本发明实施例提供了一种药品识别方法、装置和用于药品识别的装置。其中的方法具体包括：通过摄像头,获取目标药品的药盒图像；对所述药盒图像进行文本识别,以得到所述药盒图像中的药品名称；将所述药品名称作为检索词,搜索得到所述药品名称对应的药品相关信息；所述药品相关信息至少包括如下任意一项：主治病症、服用方法、说明书、不良反应、饮食禁忌；输出所述药品名称和药品相关信息。本发明实施例可以提高查找药品的效率。(The embodiment of the invention provides a medicine identification method and device and a device for medicine identification. The method specifically comprises the following steps: acquiring a medicine box image of a target medicine through a camera; performing text recognition on the medicine box image to obtain a medicine name in the medicine box image; searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications; and outputting the medicine name and the medicine related information. The embodiment of the invention can improve the efficiency of searching for the medicine.)

1. A method for identifying a medication, the method comprising:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

2. The method of claim 1, wherein the text recognition of the cartridge image to obtain the drug name in the cartridge image comprises:

performing text region detection on the medicine box image to determine a text region in the medicine box image;

identifying text information in the text region by an optical character recognition technique;

and analyzing the typesetting and semantic meaning of the text information, and extracting the medicine name from the text information.

3. The method of claim 1, further comprising:

responding to a marking operation aiming at the medicine box image, and displaying marking information corresponding to the marking operation on the medicine box image;

and storing the marked medicine box image.

4. The method of claim 1, wherein after searching for the drug related information corresponding to the drug name, the method further comprises:

determining the administration information corresponding to the medicine name according to the administration method corresponding to the medicine name, wherein the administration information comprises: the taking time and the taking dosage;

generating a medicine taking list according to the taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and sending out a prompt message when the administration time in the administration list is reached.

5. The method of claim 4, wherein after issuing the prompting message, the method further comprises:

detecting whether a medicine taken message returned aiming at a prompt message is received or not within a preset time period after the prompt message is sent out;

and recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

6. The method of claim 1, wherein the number of target drugs is greater than 1; the text recognition of the medicine box image to obtain the medicine name in the medicine box image comprises the following steps:

performing text recognition on the medicine box images respectively corresponding to the at least two target medicines to obtain the medicine name in each medicine box image as the medicine name of the target medicine corresponding to the medicine box image;

after obtaining the drug name of the target drug corresponding to each drug box image, the method further comprises:

performing conflict detection on the medicine names of the at least two target medicines to determine contraindication information of the same taking between the at least two target medicines;

and outputting the contraindication information of the same taking between the at least two target medicines.

7. The method of claim 1, wherein the outputting the drug name and drug-related information comprises:

outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to a preset multiple.

8. A drug identification device, the device comprising:

the image acquisition module is used for acquiring a medicine box image of the target medicine through the camera;

the text recognition module is used for performing text recognition on the medicine box image to obtain the medicine name in the medicine box image;

the information retrieval module is used for searching and obtaining medicine related information corresponding to the medicine name by taking the medicine name as a retrieval word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and the information output module is used for outputting the medicine name and the related information of the medicine.

9. An apparatus for drug identification comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors comprises instructions for:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

10. A machine-readable medium having stored thereon instructions, which when executed by one or more processors, cause an apparatus to perform a method of drug identification as claimed in one or more of claims 1 to 7.

Technical Field

The invention relates to the technical field of computers, in particular to a medicine identification method and device and a device for medicine identification.

Background

With the rapid development of the pharmaceutical industry, the medicines on the market are various, wherein different specifications and different dosages of the same medicine have various forms.

In daily life, a large amount of various medicines required in daily life are generally stored in a user's home. When a user is ill, the user needs to search for currently required medicines from all the medicines at home, and if the user is not familiar with the medicines, the user needs to look over the medicine descriptions on each medicine box one by one to search for the required medicines, so that the searching efficiency is low.

In addition, for patients with older age or who are not literate, the difficulty of searching for the medicine is increased, and the situation of taking the medicine by mistake is possibly caused, so that the treatment effect of the patient is possibly influenced, and other injuries are possibly caused to the body of the patient.

Disclosure of Invention

The embodiment of the invention provides a medicine identification method and device and a device for medicine identification, which can improve the efficiency of searching for medicines.

In order to solve the above problem, an embodiment of the present invention discloses a drug identification method, including:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

On the other hand, the embodiment of the invention discloses a medicine identification device, which comprises:

the image acquisition module is used for acquiring a medicine box image of the target medicine through the camera;

the text recognition module is used for performing text recognition on the medicine box image to obtain the medicine name in the medicine box image;

the information retrieval module is used for searching and obtaining medicine related information corresponding to the medicine name by taking the medicine name as a retrieval word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and the information output module is used for outputting the medicine name and the related information of the medicine.

In yet another aspect, an embodiment of the present invention discloses an apparatus for drug identification, including a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by the one or more processors includes instructions for:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

In yet another aspect, embodiments of the present invention disclose a machine-readable medium having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform one or more of the drug identification methods described above.

The embodiment of the invention has the following advantages:

according to the embodiment of the invention, a medicine box image of a target medicine is acquired in real time through a camera, and text recognition is carried out on the medicine box image to obtain the name of the medicine in the medicine box image; searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications; and finally, outputting the medicine name and the medicine related information. According to the embodiment of the invention, the medicine name and the medicine related information of the target medicine can be automatically identified and output only by scanning the target medicine through the camera of the intelligent terminal. The operation process that the user checks the medicine specification on the medicine box can be avoided, the user can be helped to quickly find the required medicine, and the efficiency that the user obtains the medicine information and finds the medicine is improved. In addition, to the patient of older age or not many literacy, reduced the degree of difficulty of medicine discernment and look for the medicine, can avoid appearing taking wrong circumstances of medicine and take place, help the user to take medicine safely.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

FIG. 1 is a flow chart of the steps of one embodiment of a method of drug identification of the present invention;

FIG. 2 is a block diagram of a drug identification device according to an embodiment of the present invention;

FIG. 3 is a block diagram of an apparatus 800 for drug identification of the present invention; and

fig. 4 is a schematic diagram of a server in some embodiments of the invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Method embodiment

Referring to fig. 1, a flowchart illustrating steps of an embodiment of a method for identifying a drug of the present invention is shown, which may specifically include the following steps:

101, acquiring a medicine box image of a target medicine through a camera;

step 102, performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

step 103, searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and 104, outputting the medicine name and the relevant information of the medicine.

The drug identification method provided by the embodiment of the invention can be applied to, but not limited to, applications with drug identification functions, such as medical-related websites and medical-related APPs (applications), and can also be applied to, but not limited to, search APPs. The medicine identification method provided by the embodiment of the invention can be applied to application environments corresponding to the client and the server, wherein the client and the server can be positioned in a wired or wireless network, and the client and the server can perform data interaction through the wired or wireless network.

Specifically, the client may run on an intelligent terminal, and the intelligent terminal may specifically include but is not limited to: smart phones, tablet computers, electronic book readers, MP3 (motion picture Experts Group Audio Layer III) players, MP4 (motion picture Experts Group Audio Layer IV) players, laptop portable computers, car-mounted computers, desktop computers, set-top boxes, smart televisions, wearable devices, and the like.

In the embodiment of the invention, the user can trigger the medicine identification function through the preset interface provided by the client. For example, by triggering a "medicine identification" button in the APP, a medicine identification request may be generated, and in response to the medicine identification request, the client may start a camera of the intelligent terminal, and obtain, in real time, a medicine box image of the target medicine through the camera, for example, a photo or a video of the medicine box image containing the target medicine may be taken. Or, the embodiment of the present invention may further automatically execute the medicine identification method according to the embodiment of the present invention when it is detected that the camera is started. For example, whether a medicine box image is contained in a picture acquired by the camera can be detected in real time, if yes, the medicine box image is determined to be the medicine box image of the target medicine to be identified, and subsequent identification operation can be executed. It can be understood that in the embodiment of the present invention, the photo or the video of the medicine box image taken by the camera may be recognized, and the name of the medicine is extracted from the medicine box image included in the photo or the video, or, in the case that the medicine box image is obtained by the camera, the medicine box image obtained by the camera may be directly recognized in real time without taking the photo or the video, and the name of the medicine therein is extracted.

Specifically, after acquiring the medicine box image, text recognition may be performed on the medicine box image to obtain a medicine name in the medicine box image. In an optional embodiment of the present invention, the step S102 of performing text recognition on the medicine box image to obtain a name of a medicine in the medicine box image may specifically include:

step S11, text region detection is carried out on the medicine box image to determine the text region in the medicine box image;

step S12, recognizing text information in the text area through an Optical Character Recognition (OCR) technique;

and step S13, analyzing the typesetting and semantic meaning of the text information, and extracting the medicine name from the text information.

In a specific application, the medicine box usually contains picture information and text information, and the text information generally comprises a medicine name, a manufacturer, a batch number, a taking method and the like. The medicine names are usually located in a more striking area in the medicine box, and occupy one row separately, and the font ratio is larger.

According to the display characteristics of the medicine names in the medicine boxes, the embodiment of the invention firstly detects the text areas of the medicine box images to determine the text areas in the medicine box images, then identifies the text information in the text areas through an OCR technology, analyzes the typesetting and semantic meaning of the text information, and extracts the medicine names from the text information. For example, for a plurality of extracted text regions, the position of each text region in the medicine box and the size of the characters in the text region are analyzed, and the text information in the text region which is located at the relatively middle position of the medicine box and has a relatively large font is taken as the medicine name.

It is to be understood that the embodiment of the present invention does not limit the specific manner of determining the text region in the medicine box image. For example, the kit image may be thresholded to determine text regions in the kit image. Specifically, the thresholding process may select different thresholding methods according to actual conditions, such as a fixed thresholding method, an adaptive thresholding method, the Otsu method, an iterative method, and the like. Alternatively, the text region in the cartridge image may also be determined from the convolutional neural network.

Optionally, in the embodiment of the present invention, the identified name of the drug may be retrieved to determine whether the identified name of the drug is accurate, and if the retrieval result does not include the drug information corresponding to the name of the drug, the image of the drug cassette may be re-identified to re-identify the name of the drug therein.

After the medicine name of the target medicine is identified and obtained, the medicine name can be used as a search word, and medicine related information corresponding to the medicine name can be obtained through searching. For example, a medical information retrieval system may be incorporated to retrieve the drug-related information corresponding to the drug name, such as common names, drug ingredients, medication categories, indications, usage amounts, instructions, etc. Optionally, according to the name of the medicine, the embodiment of the present invention may further obtain the cautionary matters, the adverse reactions, and the like when the medicine is taken from the searched medical related information such as the medicine instruction book, the doctor question and answer, the authoritative article, and the like. For example, some medicines may stimulate the intestines and stomach and the like and need to be taken after a meal, while some medicines may be suitable to be taken before a meal, and the information related to the medicines of the embodiment of the present invention may further include attention information and the like when the medicines are taken, for example, before or after a meal.

The medicine classification can be classified according to the function of the medicine, such as endocrine medicine, hypotensive medicine, etc.; it can also be classified according to whether it is a prescription drug, such as a prescription drug, an over-the-counter drug; or, the Chinese medicine or the western medicine can be classified and the like.

The related information usually needs the user to check the instruction manual or perform related retrieval for understanding, but the embodiment of the invention can automatically search and obtain the related information of the medicine according to the identified medicine name and feed the related information back to the user, so that the time for the user to check the instruction manual and retrieve can be saved.

In an optional embodiment of the present invention, the identified medicine name may be input into a GIS (Geographic Information System) to search for pharmacy Information for which a medicine corresponding to the medicine name can be purchased nearby, a button for triggering display of the pharmacy Information may be provided on the identification interface, and when a triggering operation of the button by the user is received, Information such as the name and address of a pharmacy for which the medicine can be purchased nearby may be displayed.

After the medicine name and the medicine related information of the target medicine are acquired, the medicine name and the medicine related information can be output in a voice or text mode and the like.

In an optional embodiment of the present invention, the outputting the name of the drug and the related information of the drug may specifically include:

outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to a preset multiple.

For the medicine name and the medicine related information obtained by recognition, the embodiment of the invention can output the medicine name and the medicine related information in any form such as voice and/or text after the recognition is finished so as to show the medicine name and the medicine related information to the user. For users who do not clearly hear during voice broadcasting, the medicine names and the medicine related information in the text form can be checked so as to meet the diversified requirements of the users.

Optionally, the name of the medicine and the text information of the medicine-related information may be displayed in a display interface of a medicine box picture of the target medicine. For example, the display screen of the intelligent terminal can be split to obtain a first screen and a second screen, a medicine box image is displayed in the first screen, a medicine name and medicine related information corresponding to the medicine box image are displayed in the second screen, and then the corresponding relation between the medicine name and the medicine related information and a target medicine can be visually displayed for a user.

In addition, the embodiment of the invention can amplify and display the medicine name and the text information corresponding to the medicine related information according to the preset times, and the old user with poor eyesight can more clearly know the medicine related information. It can be understood that the preset multiple can be obtained through setting operation of a user, and in specific implementation, the preset multiple can be further adjusted through a zooming gesture of the user.

In an alternative embodiment of the present invention, the number of target drugs may be greater than 1; the text recognition of the medicine box image to obtain the medicine name in the medicine box image may specifically include:

performing text recognition on the medicine box images respectively corresponding to the at least two target medicines to obtain the medicine name in each medicine box image as the medicine name of the target medicine corresponding to the medicine box image;

after obtaining the drug name of the target drug corresponding to each drug box image, the method may further include:

step S21, performing conflict detection on the medicine names of the at least two target medicines to determine contraindication information of the same taking between the at least two target medicines;

and step S22, outputting contraindication information of the same taking between the at least two target medicines.

In specific application, a user may need to take multiple medicines at the same time, however, mutual exclusion may exist between some medicines, for example, the Chinese patent medicine and the western medicine cannot be taken simultaneously, the neostigmine bromide and the penicillins cannot be taken simultaneously, the cold medicines cannot be taken simultaneously, and the like. If mutually exclusive medicines are taken simultaneously, the user body may be hurt or the taking effect of the medicines may be influenced.

In order to solve the above problem, in the embodiment of the present invention, the cartridge images corresponding to at least two target drugs may be obtained, and text recognition is performed on the cartridge image of the cartridge corresponding to each target drug in the at least two target drugs, so as to obtain a drug name in each cartridge image, which is used as the drug name of the target drug corresponding to the cartridge image. Further, the conflict detection can be performed on the at least two target products, for example, whether the medicines are contraindicated for taking the medicines together can be judged according to the medicine names of the at least two target medicines by combining a search technology, and if the contraindicated for taking the medicines together exist, contraindicated information for taking the medicines together can be output to prompt a user that the medicines are not suitable for taking the medicines together, so that the safe medication of the user is facilitated. Wherein the contraindication information of the same-taking may include: there are names of medicines that are contraindicated for taking the medicines together, which adverse effects may be caused by taking the medicines together, safety methods for taking the medicines together, and the like.

In an optional embodiment of the invention, the method may further comprise:

step S31, responding to the marking operation of the medicine box image, and displaying the marking information corresponding to the marking operation on the medicine box image;

and step S32, storing the marked medicine box image.

For the cartridge image of the target drug taken by the user, the embodiment of the present invention may further display the marking information corresponding to the marking operation on the cartridge image. For example, in an identification interface displaying a name of a medicine and information related to the medicine, a button for triggering execution of a marking operation may be displayed, and when a triggering operation of a user on the button is received, a medicine box image displayed in the current identification interface may be marked. Alternatively, the user may select a stored image of the medicine box from the album to mark the image.

The marking information may be manually added by the user according to a marking operation performed by the user after acquiring the cartridge image of the target medicine. Alternatively, the marking information may be added automatically according to the identification content after the medicine box image of the target medicine is identified. The identification content may include a name of a drug in the identified image of the drug cassette, information related to the drug corresponding to the name of the drug, and the like.

It is to be understood that the content and form of the marking information are not limited by the embodiments of the present invention, for example, the marking information may further include information such as an expiration time of the target drug. The marking information can be in any form of text, graphics, pictures and the like.

In the embodiment of the invention, the marked information of the medicine is pasted on the medicine box image by an AR (Augmented Reality) technology, and the marked medicine box image is stored in a local album, so that a user can conveniently check the medicine box image in the future, for example, for the medicine with the name of metformin, the following marked information can be added on the medicine box image corresponding to the medicine: hypotensive drugs (prohibited to be taken on empty stomach).

In an optional embodiment of the present invention, after obtaining the medicine related information corresponding to the medicine name by searching, the method may further include:

step S41, determining, according to the medication method corresponding to the medicine name, medication information corresponding to the medicine name, where the medication information includes: the taking time and the taking dosage;

step S42, generating a medicine taking list according to the medicine taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and a step S43, when the administration time in the administration list is up, sending out a prompt message.

In a specific application, a patient with various diseases may need to take a plurality of medicines at the same time, however, due to different taking time and taking dosage of different medicines, the situations of taking medicine on time, taking excessive medicine or taking wrong medicine frequently occur.

In order to solve the above problems, in the embodiments of the present invention, a medicine taking list may be generated according to a plurality of medicine box images of target medicines taken by a user, and the user may clearly know the medicine to be taken and the medicine taking method by looking up the medicine taking list without looking up the description on the medicine box each time, which is convenient for the user. In addition, the embodiment of the invention can automatically send out a prompt message according to the taking time of each medicine in the medicine taking list so as to remind the user to take the medicine on time, thereby avoiding the situation that the user forgets to take the medicine.

It is to be understood that the display form of the medicine taking list in the embodiment of the present invention is not limited, and for example, the medicine taking list may be displayed in a list or a table form. Referring to table 1, a specific illustration of a dosage list of the present invention is shown.

TABLE 1

Name of medicine	Time of administration	Dosage to be taken
			White plus black	In the morning	White slice 1 slice
Youmeile	In the morning	0.5 tablet
			White plus black	Noon is a Chinese traditional musical instrument	White slice 1 slice
White plus black	At night	Black tablet 1
			Neostigmine	At night	1 piece of

Under the condition that the taking time in the taking list is reached, the client can send out a prompt message to remind the user to take the medicine on time. In the embodiment of the invention, the default reminding time for sending the prompting message can be automatically generated according to the taking time of the medicine. For example, if the administration time is morning, the default reminder time may be set to 8 am, if the administration time is noon, the default reminder time may be set to 12 am, and if the administration time is evening, the default reminder time may be set to 8 pm.

Optionally, the embodiment of the present invention may further provide a setting interface for setting the reminding time of each item of medicine to the user, and the user may modify the default reminding time through the setting interface. Referring to table 2, a specific illustration of another dosage list of the present invention is shown. As shown in table 2, the reminding time is set in the medicine taking list, and the medicine taking list with the reminding time can send a prompt message when the reminding time arrives.

TABLE 2

Name of medicine	Time of administration	Reminding time	Dosage to be taken
				White plus black	In the morning	7：00	White slice 1 slice
Youmeile	In the morning	7：30	0.5 tablet
				White plus black	Noon is a Chinese traditional musical instrument	12：00	White slice 1 slice
White plus black	At night	8：00	Black tablet 1
				Neostigmine	At night	8：30	1 piece of

Optionally, when the administration time or the reminding time arrives, the client may ring a preset ring to remind the user of the arrival of the administration time, and may output a prompt message in a form of text and/or voice broadcast, where the prompt message at least may include any one of the following: the medicine list comprises medicine names of the arrival of the administration time in the administration list, medicine related information corresponding to the medicine names of the arrival of the administration time in the administration list, medicine box images corresponding to the medicine names of the arrival of the administration time in the administration list and the administration list.

For example, the name of a medicine whose administration time reaches in the administration list may be broadcasted in a voice, information related to the medicine corresponding to the medicine name, such as an administration method, an administration dosage, and the like, may be broadcasted in a voice, and an administration notice corresponding to the medicine name, such as that a user cannot drink a cephalo-type medicine, and the like, may be broadcasted in a voice.

In addition, the embodiment of the invention can also take the medicine box image corresponding to the name of the medicine which arrives at the administration time in the administration list as the prompt information to show to the user, so that the user can conveniently and quickly find the medicine to be taken through the medicine box image.

Optionally, in the embodiment of the present invention, a voice template, a sentence template, and the like of the voice broadcast may also be customized, for example: when the medicine is taken, people can take the white and black tablets one by one and take the medicine on time to get the effect early! ". It can be understood that the embodiment of the present invention does not limit the specific prompting manner of the prompting message. Optionally, on the basis of the voice prompt, a pop-up window prompt can be added, and information of the medicine which needs to be taken currently is displayed in the pop-up window.

Optionally, in the embodiment of the present invention, a button for triggering execution of an operation of adding a medication list may be displayed in an identification interface displaying a name of a medicine and related information of the medicine, and when a trigger operation of a user for the button is received, information such as a time of taking a medicine and a dose of taking the medicine may be extracted from the related information of the medicine in the current identification interface, and further, the name of the medicine, the time of taking the medicine corresponding to the name of the medicine, and the dose of taking the medicine may be added to an existing medication list.

Optionally, in the embodiment of the present invention, after receiving the medicine box image corresponding to the target medicine shot by the user, the user may be asked whether to add the currently shot medicine box image to an existing medicine taking list, and if the user selects to add, the name and the administration information of the medicine in the currently shot medicine box image may be identified, and the medicine name and the administration information are merged with the existing medicine taking list to obtain a merged medicine taking list, so that the user may manage information of a plurality of medicines to be taken through one medicine taking list, thereby providing more convenience for the user.

In an optional embodiment of the present invention, after issuing the prompt message, the method may further include:

step S51, detecting whether a medicine taken message returned by the prompt message is received within a preset time period after the prompt message is sent out;

and step S52, recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

The embodiment of the invention can also provide a medicine taking recording function, and whether a medicine taken message returned by a user aiming at the prompt message is received or not is detected in a preset time period after the prompt message is sent each time, if so, the taking state of the medicine corresponding to the prompt message can be recorded as taken, and the taken time is recorded, otherwise, the taking state of the medicine corresponding to the prompt message can be recorded as not taken, and the not-taken time is recorded. By recording the taking state of each medicine in the medicine taking list of the user in a period of time, a medicine taking record list can be generated to help the user to know the medicine taking condition of the user.

In summary, in the embodiment of the present invention, a medicine box image of a target medicine is acquired through a camera of an intelligent terminal, and text recognition is performed on the medicine box image to obtain a medicine name in the medicine box image; searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications; and finally, outputting the medicine name and the medicine related information. According to the embodiment of the invention, the medicine name and the medicine related information of the target medicine can be automatically identified and output only by scanning the target medicine through the camera of the intelligent terminal. The operation process that the user checks the medicine specification on the medicine box can be avoided, the user can be helped to quickly find the required medicine, and the efficiency that the user obtains the medicine information and finds the medicine is improved. In addition, to the patient of older age or not many literacy, reduced the degree of difficulty of medicine discernment and look for the medicine, can avoid appearing taking wrong circumstances of medicine and take place, help the user to take medicine safely.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Device embodiment

Referring to fig. 2, a block diagram of a medicine identification device according to an embodiment of the present invention is shown, where the device may specifically include:

the image acquisition module 201 is used for acquiring a medicine box image of a target medicine through a camera;

the text recognition module 202 is configured to perform text recognition on the medicine box image to obtain a medicine name in the medicine box image;

the information retrieval module 203 is configured to search for medicine related information corresponding to the medicine name by using the medicine name as a retrieval word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and the information output module 204 is configured to output the medicine name and the medicine related information.

Optionally, the text recognition module 202 may specifically include:

the region detection submodule is used for detecting the text region of the medicine box image so as to determine the text region in the medicine box image;

the text recognition sub-module is used for recognizing the text information in the text area through an optical character recognition technology;

and the name extraction submodule is used for analyzing the typesetting and the semantic meaning of the text information and extracting the medicine name from the text information.

Optionally, the apparatus may further include:

the marking module is used for responding to a marking operation aiming at the medicine box image and displaying marking information corresponding to the marking operation on the medicine box image;

and the storage module is used for storing the marked medicine box images.

Optionally, the apparatus may further include:

an information extraction module, configured to determine, according to a taking method corresponding to the medicine name, taking information corresponding to the medicine name, where the taking information includes: the taking time and the taking dosage;

the list generation module is used for generating a medicine taking list according to the taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and the message prompting module is used for sending a prompting message under the condition that the taking time in the taking list is reached.

Optionally, the apparatus may further include:

the message detection module is used for detecting whether a medicine taken message returned aiming at the prompt message is received or not within a preset time period after the prompt message is sent out;

and the state recording module is used for recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

Optionally, the number of target drugs is greater than 1; the text recognition module may be specifically configured to perform text recognition on the cartridge images corresponding to the at least two target drugs respectively to obtain a drug name in each cartridge image, where the drug name is used as a drug name of the target drug corresponding to the cartridge image;

the apparatus may further include:

the conflict detection module is used for carrying out conflict detection on the medicine names of the at least two target medicines so as to determine the contraindication information of the same taking between the at least two target medicines;

and the contraindication output module is used for outputting the same-taking contraindication information between the at least two target medicines.

Optionally, the information output module 204 may specifically include:

the first output submodule is used for outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And the second output submodule is used for amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to the preset times.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

An embodiment of the present invention provides an apparatus for drug identification, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs configured to be executed by one or more processors include instructions for: acquiring a medicine box image of a target medicine through a camera; performing text recognition on the medicine box image to obtain a medicine name in the medicine box image; searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications; and outputting the medicine name and the medicine related information.

Fig. 3 is a block diagram illustrating an apparatus 800 for drug identification according to an example embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 3, the apparatus 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power components 806 provide power to the various components of device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 800.

The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice information processing mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or a component of the apparatus 800, the presence or absence of user contact with the apparatus 800, orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast drug-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Fig. 4 is a schematic diagram of a server in some embodiments of the invention. The server 1900, which may vary widely in configuration or performance, may include one or more Central Processing Units (CPUs) 1922 (e.g., one or more processors) and memory 1932, one or more storage media 1930 (e.g., one or more mass storage devices) storing applications 1942 or data 1944. Memory 1932 and storage medium 1930 can be, among other things, transient or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown), each of which may include a series of instructions operating on a server. Still further, a central processor 1922 may be provided in communication with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.

The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input-output interfaces 1958, one or more keyboards 1956, and/or one or more operating systems 1941, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

A non-transitory computer readable storage medium in which instructions, when executed by a processor of an apparatus (server or terminal), enable the apparatus to perform the drug identification method shown in fig. 1.

A non-transitory computer readable storage medium in which instructions, when executed by a processor of an apparatus (server or terminal), enable the apparatus to perform a method of drug identification, the method comprising: acquiring a medicine box image of a target medicine through a camera; performing text recognition on the medicine box image to obtain a medicine name in the medicine box image; searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications; and outputting the medicine name and the medicine related information.

The embodiment of the invention discloses an A1 medicine identification method, which comprises the following steps:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

A2, according to the method in A1, the text recognition of the medicine box image to obtain the medicine name in the medicine box image comprises:

performing text region detection on the medicine box image to determine a text region in the medicine box image;

identifying text information in the text region by an optical character recognition technique;

and analyzing the typesetting and semantic meaning of the text information, and extracting the medicine name from the text information.

A3, the method of A1, the method further comprising:

responding to a marking operation aiming at the medicine box image, and displaying marking information corresponding to the marking operation on the medicine box image;

and storing the marked medicine box image.

A4, according to the method in A1, after searching for the medicine-related information corresponding to the medicine name, the method further includes:

determining the administration information corresponding to the medicine name according to the administration method corresponding to the medicine name, wherein the administration information comprises: the taking time and the taking dosage;

generating a medicine taking list according to the taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and sending out a prompt message when the administration time in the administration list is reached.

A5, according to the method of A4, after issuing the prompting message, the method further comprising:

detecting whether a medicine taken message returned aiming at a prompt message is received or not within a preset time period after the prompt message is sent out;

and recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

A6, the number of target drugs according to the method of A1 is greater than 1; the text recognition of the medicine box image to obtain the medicine name in the medicine box image comprises the following steps:

performing text recognition on the medicine box images respectively corresponding to the at least two target medicines to obtain the medicine name in each medicine box image as the medicine name of the target medicine corresponding to the medicine box image;

after obtaining the drug name of the target drug corresponding to each drug box image, the method further comprises:

performing conflict detection on the medicine names of the at least two target medicines to determine contraindication information of the same taking between the at least two target medicines;

and outputting the contraindication information of the same taking between the at least two target medicines.

A7, the method of A1, the outputting the drug name and drug related information, comprising:

outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to a preset multiple.

The embodiment of the invention discloses B8 and a medicine identification device, which comprises:

the image acquisition module is used for acquiring a medicine box image of the target medicine through the camera;

the text recognition module is used for performing text recognition on the medicine box image to obtain the medicine name in the medicine box image;

the information retrieval module is used for searching and obtaining medicine related information corresponding to the medicine name by taking the medicine name as a retrieval word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and the information output module is used for outputting the medicine name and the related information of the medicine.

B9, the apparatus of B8, the text recognition module comprising:

the region detection submodule is used for detecting the text region of the medicine box image so as to determine the text region in the medicine box image;

the text recognition sub-module is used for recognizing the text information in the text area through an optical character recognition technology;

and the name extraction submodule is used for analyzing the typesetting and the semantic meaning of the text information and extracting the medicine name from the text information.

B10, the apparatus of B8, the apparatus further comprising:

the marking module is used for responding to a marking operation aiming at the medicine box image and displaying marking information corresponding to the marking operation on the medicine box image;

and the storage module is used for storing the marked medicine box images.

B11, the apparatus of B8, the apparatus further comprising:

an information extraction module, configured to determine, according to a taking method corresponding to the medicine name, taking information corresponding to the medicine name, where the taking information includes: the taking time and the taking dosage;

the list generation module is used for generating a medicine taking list according to the taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and the message prompting module is used for sending a prompting message under the condition that the taking time in the taking list is reached.

B12, the apparatus of B11, the apparatus further comprising:

the message detection module is used for detecting whether a medicine taken message returned aiming at the prompt message is received or not within a preset time period after the prompt message is sent out;

and the state recording module is used for recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

B13, the device according to B8, the number of the target drugs is more than 1; the text recognition module is specifically configured to perform text recognition on the medicine box images corresponding to the at least two target medicines respectively to obtain a medicine name in each medicine box image, and the medicine name is used as the medicine name of the target medicine corresponding to the medicine box image;

the device further comprises:

the conflict detection module is used for carrying out conflict detection on the medicine names of the at least two target medicines so as to determine the contraindication information of the same taking between the at least two target medicines;

and the contraindication output module is used for outputting the same-taking contraindication information between the at least two target medicines.

B14, the apparatus according to B8, the information output module comprising:

the first output submodule is used for outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And the second output submodule is used for amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to the preset times.

The embodiment of the invention discloses C15, a device for drug identification, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs configured to be executed by the one or more processors comprise instructions for:

acquiring a medicine box image of a target medicine through a camera;

performing text recognition on the medicine box image to obtain a medicine name in the medicine box image;

searching to obtain medicine related information corresponding to the medicine name by taking the medicine name as a search word; the related information of the medicine at least comprises any one of the following items: indications, administration methods, instructions, adverse reactions, dietary contraindications;

and outputting the medicine name and the medicine related information.

C16, the text recognition of the medicine box image according to the apparatus of C15 to obtain the medicine name in the medicine box image, including:

performing text region detection on the medicine box image to determine a text region in the medicine box image;

identifying text information in the text region by an optical character recognition technique;

and analyzing the typesetting and semantic meaning of the text information, and extracting the medicine name from the text information.

C17, the device of C15, the device also configured to execute the one or more programs by one or more processors including instructions for:

responding to a marking operation aiming at the medicine box image, and displaying marking information corresponding to the marking operation on the medicine box image;

and storing the marked medicine box image.

C18, the device of C15, the device also configured to execute the one or more programs by one or more processors including instructions for:

determining the administration information corresponding to the medicine name according to the administration method corresponding to the medicine name, wherein the administration information comprises: the taking time and the taking dosage;

generating a medicine taking list according to the taking information; the medicine taking list comprises: the corresponding relation among the medicine name, the taking time and the taking dosage;

and sending out a prompt message when the administration time in the administration list is reached.

C19, the device of C18, the device also configured to execute the one or more programs by one or more processors including instructions for:

detecting whether a medicine taken message returned aiming at a prompt message is received or not within a preset time period after the prompt message is sent out;

and recording the taking state of each item of medicine in the medicine taking list according to the received medicine taken message.

C20, the device of C15, the number of target drugs is greater than 1; the text recognition of the medicine box image to obtain the medicine name in the medicine box image comprises the following steps:

performing text recognition on the medicine box images respectively corresponding to the at least two target medicines to obtain the medicine name in each medicine box image as the medicine name of the target medicine corresponding to the medicine box image;

after obtaining the drug name in each of the cartridge images, the method further comprises:

performing conflict detection on the medicine names of the at least two target medicines to determine contraindication information of the same taking between the at least two target medicines;

and outputting the contraindication information of the same taking between the at least two target medicines.

C21, the apparatus of C15, the outputting the drug name and drug related information, comprising:

outputting the medicine name and the voice information corresponding to the medicine related information in a voice broadcasting mode; and/or

And amplifying and displaying the medicine name and the text information corresponding to the medicine related information according to a preset multiple.

Embodiments of the present invention disclose D22, a machine-readable medium having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform a method of drug identification as described in one or more of a 1-a 7.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

The above detailed descriptions of the drug identification method, the drug identification device and the device for drug identification provided by the present invention are provided, and the principle and the implementation of the present invention are explained by applying specific examples, and the descriptions of the above examples are only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.