阅读说明：本技术 一种吸头图像处理方法、装置和吸头图像处理设备 (Sucker image processing method and device and sucker image processing equipment ) 是由 林贵文 严欢 汪建德 于 2021-06-29 设计创作，主要内容包括：本发明实施例涉及一种吸头图像处理方法、装置和吸头图像处理设备,所述方法包括：获取吸头图像；对所述吸头图像进行初次二值化,得到有效区域图；对所述有效区域图进行梯度边缘检测,得到边缘检测图,且对所述边缘检测图进行梯度二值化处理,得到梯度二值化图；在所述梯度二值化图中确定存在吸头时,对所述有效区域图进行吸头特征提取,获得吸头特征,所述吸头特征包括吸头轮廓及吸头液面位置；根据所述吸头轮廓,判断所述边缘检测图中是否存在有效缺陷；若存在所述有效缺陷,则根据所述有效缺陷的特征及所述吸头液面位置判断吸样质量。本发明实施例能够判断吸头内部的吸样情况,且判断吸样质量。(The embodiment of the invention relates to a sucker image processing method, a sucker image processing device and sucker image processing equipment, wherein the method comprises the following steps: acquiring a suction head image; carrying out primary binarization on the suction head image to obtain an effective area image; performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image; when a suction head is determined to exist in the gradient binary image, carrying out suction head feature extraction on the effective area image to obtain suction head features, wherein the suction head features comprise a suction head outline and a suction head liquid level position; judging whether effective defects exist in the edge detection image or not according to the suction head profile; and if the effective defects exist, judging the sample sucking quality according to the characteristics of the effective defects and the liquid level position of the sucking head. The embodiment of the invention can judge the sample sucking condition in the suction head and judge the sample sucking quality.)

1. A tip image processing method, characterized in that the method comprises:

acquiring a suction head image;

carrying out primary binarization on the suction head image to obtain an effective area image;

performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image;

when a suction head is determined to exist in the gradient binary image, carrying out suction head feature extraction on the effective area image to obtain suction head features, wherein the suction head features comprise a suction head outline and a suction head liquid level position;

judging whether effective defects exist in the edge detection image or not according to the suction head profile;

and if the effective defects exist, judging the sample sucking quality according to the characteristics of the effective defects and the liquid level position of the sucking head.

2. The method of claim 1, further comprising:

if the sum of the pixels of the gradient binary image is larger than a first threshold value, judging that a suction head exists in the gradient binary image;

and if the sum of the pixels of the gradient binary image is not greater than the first threshold value, judging that no suction head exists in the gradient binary image.

3. The method of claim 1, wherein performing tip feature extraction on the active area map to obtain tip features, wherein the tip features include a tip profile and a tip liquid level position, and comprises:

determining the search range of the suction head root node in the gradient binary image, and searching the suction head root node in the effective area image;

searching a local gray minimum value in the effective area graph by taking the suction head root node as a starting point and in the direction of the suction head vertex and taking the pixel range of a second threshold value where the two side outlines are located to obtain the two side outlines of the suction head;

determining the top point of the suction head according to the profiles of the two sides of the suction head, determining the profile of the suction head according to the profiles of the two sides of the suction head and the top point of the suction head, and determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic that gray level mutation exists at the junction of liquid and air.

4. The method of claim 3, wherein determining a search range for tip root nodes in the gradient binarization map and searching for tip root nodes in the active area map comprises:

determining the search range of root nodes in the effective area map at the row of the suction head root position of the gradient binary map;

and finding a suction head root node in the search range in the effective area graph by using the characteristic of the local minimum value of the gray level of the root node.

5. The method of claim 3, wherein said determining a tip apex from said two-sided profile of said tip comprises:

and determining the intersection point of the two side profiles of the suction head as the top point of the suction head.

6. The method of claim 3, wherein the two-sided profile comprises an upper profile and a lower profile; the determining the liquid level position of the suction head according to the two side profiles and the characteristic that the gray level mutation exists at the junction of the liquid and the air comprises the following steps:

calculating the gray level first-order difference of the gray level sequences of the upper contour and the lower contour to obtain a gray level first-order difference sequence;

respectively calculating liquid level position candidate points of the upper contour and the lower contour according to the gray level first-order difference sequence;

if first-order differential values meeting preset conditions are found in the gray first-order differential sequences of the upper contour and the lower contour respectively, judging that the sample is sucked, wherein the preset conditions are that the first-order differential values in the gray first-order differential sequences are smaller than the point of a negative peak of a third threshold value, and the corresponding position distance of the negative peak of the upper contour or the lower contour is not more than 2;

and averaging the positions of the negative peak points of the upper contour and the lower contour in the gray first-order difference meeting the preset condition to obtain the liquid level position of the suction head.

7. The method of claim 6, wherein calculating the level position candidate points for the upper and lower contours respectively from the gray first order difference sequence comprises:

and taking the negative peak in the gray first-order difference sequence as a liquid level position candidate point of the upper contour and the lower contour.

8. The method of claim 6, further comprising:

and if the point of the negative peak sequence with the first-order difference value smaller than the third threshold value cannot be searched in the gray first-order difference sequence, judging that no sample is absorbed.

9. The method of claim 1 or 3, wherein said determining whether a valid defect exists in said edge detection map based on said tip profile comprises:

removing the suction head contour from the edge detection image to obtain a defect contour image;

and calculating the area of each contour in the defect contour map, and if the contour with the contour area not smaller than a fifth threshold exists in the defect contour map, judging that the defect contour map has effective defects.

10. The method of claim 9, further comprising:

and if the contour with the contour area smaller than the fifth threshold exists in the defect contour map, judging the contour with the contour area smaller than the fifth threshold as an invalid defect, and after removing the invalid defect, judging that the sample suction is qualified if the contour with the contour area not smaller than the fifth threshold does not exist.

11. The method of claim 9, wherein said removing the tip profile from the edge detection map resulting in a defect profile map comprises:

in the edge detection map, adjacent pixel values within a fourth threshold range of the suction head profile are set to be 0, and a defect profile map is obtained.

12. The method of claim 9, wherein said determining the quality of the draw based on the characteristics of the valid defects and the tip level position comprises:

marking the valid defects;

if the center of the effective defect is outside the contour of the suction head or is at a position other than a position between the liquid level position of the suction head and the top point of the suction head, judging that the sample is hung on the wall;

calculating the defect characteristics of the effective defects if the centers of the effective defects are in the inner part of the outline of the suction head or in the position between the liquid level position of the suction head and the top point of the suction head;

and if the roundness of the defect outline corresponding to the defect characteristic is larger than the fifth threshold, judging the defect outline to be a bubble, otherwise, judging the defect outline to be a sample wall hanging.

13. The method of any one of claims 1 to 12, wherein after said acquiring a tip image, the method further comprises:

and carrying out filtering processing on the sucker image.

14. The method of claim 13, wherein the primary binarization of the tip image to obtain an effective area map comprises:

carrying out primary binarization on the suction head image to obtain a binary image;

carrying out pixel summation on the binary image in the horizontal direction and the vertical direction respectively to obtain a horizontal projection sequence and a vertical projection sequence;

respectively searching a starting position and a terminal position which are not zero in the horizontal projection sequence and the vertical projection sequence to be used as the boundary of the effective area of the sucker image;

and obtaining the effective area map according to the boundary of the effective area of the sucker image.

15. A tip image processing apparatus characterized by comprising:

the acquisition module is used for acquiring a sucker image;

the binarization module is used for carrying out primary binarization on the suction head image to obtain an effective area map;

the gradient detection module is used for carrying out gradient edge detection on the effective region map to obtain an edge detection map, and carrying out gradient binarization processing on the edge detection map to obtain a gradient binarization map;

the characteristic extraction module is used for extracting the characteristics of the suction head from the effective area map to obtain the characteristics of the suction head when the suction head is determined to exist in the gradient binary image, wherein the characteristics of the suction head comprise the contour of the suction head and the liquid level position of the suction head;

the defect judging module is used for judging whether effective defects exist in the edge detection image according to the suction head profile;

and the sample suction quality judging module is used for judging the sample suction quality according to the characteristics of the effective defects and the liquid level position of the suction head if the effective defects exist.

16. A tip image processing apparatus characterized by comprising:

at least one processor, and

a memory communicatively coupled to the processor, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method of any of claims 1-14.

17. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a tip image processing device, cause the tip image processing device to perform the method of any one of claims 1-14.

Technical Field

The embodiment of the invention relates to the technical field of medical equipment, in particular to a sucker image processing method and device and sucker image processing equipment.

Background

The nucleic acid detection process can be roughly divided into three steps: sample collection, nucleic acid extraction, PCR amplification and analysis. After the sample collection is completed, the sample to be detected in the virus sampling tube needs to be subpackaged into a deep-hole plate for subsequent nucleic acid extraction.

The existing automatic cup separating system can reduce the contact of experimenters and virus samples, and reduce the error probability and the biological exposure risk.

However, the conductive pipette tip adopted by the existing automatic cup separating system can realize the subpackage processing of the virus sample, but cannot judge the sample sucking condition in the pipette tip in the subpackage process.

Disclosure of Invention

The invention aims to provide a sucker image processing method, a sucker image processing device and sucker image processing equipment, which can judge the sample sucking condition in a sucker and judge the sample sucking quality.

In a first aspect, an embodiment of the present invention provides a tip image processing method, where the method includes:

acquiring a suction head image;

carrying out primary binarization on the suction head image to obtain an effective area image;

performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image;

when a suction head is determined to exist in the gradient binary image, carrying out suction head feature extraction on the effective area image to obtain suction head features, wherein the suction head features comprise a suction head outline and a suction head liquid level position;

judging whether effective defects exist in the edge detection image or not according to the suction head profile;

and if the effective defects exist, judging the sample sucking quality according to the characteristics of the effective defects and the liquid level position of the sucking head.

In some embodiments, the method further comprises:

if the sum of the pixels of the gradient binary image is larger than a first threshold value, judging that a suction head exists in the gradient binary image;

and if the sum of the pixels of the gradient binary image is not greater than the first threshold value, judging that no suction head exists in the gradient binary image.

In some embodiments, the performing a tip feature extraction on the active area map to obtain a tip feature, where the tip feature includes a tip profile and a tip liquid level position, includes:

determining a suction head root node searching range in the gradient binary image, and searching a suction head root node in the effective area image;

searching a local gray minimum value in the effective area graph by taking the suction head root node as a starting point and in the direction of the suction head vertex and taking the pixel range of a second threshold value where the two side outlines are located to obtain the two side outlines of the suction head;

determining the top point of the suction head according to the profiles of the two sides of the suction head, determining the profile of the suction head according to the profiles of the two sides of the suction head and the top point of the suction head, and determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic that gray level mutation exists at the junction of liquid and air.

In some embodiments, the determining a tip root node search range in the gradient binarization map and searching a root node in the effective area map comprises:

determining the search range of root nodes in the effective area map at the row of the suction head root position of the gradient binary map;

and finding a suction head root node in the search range of the effective area graph by using the characteristic of the gray local minimum of the root node.

In some embodiments, said determining a tip apex from said two-sided profile of said tip comprises:

determining the intersection point of the profiles of the two sides of the suction head as the top point of the suction head;

in some embodiments, the two-sided profile includes an upper profile and a lower profile; the method for determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic that the gray level mutation exists at the junction of liquid and air comprises the following steps:

calculating the gray level first-order difference of the gray level sequences of the upper contour and the lower contour to obtain a gray level first-order difference sequence;

respectively calculating liquid level position candidate points of the upper contour and the lower contour according to the gray level first-order difference sequence;

if first-order differential values meeting preset conditions are found in the gray first-order differential sequences of the upper contour and the lower contour respectively, judging that the sample is sucked, wherein the preset conditions are that the first-order differential values in the gray first-order differential sequences are smaller than the point of a negative peak of a third threshold value, and the corresponding position distance of the negative peak of the upper contour or the lower contour is not more than 2;

and averaging the positions of the negative peak points of the upper contour and the lower contour in the gray first-order difference meeting the preset condition to obtain the liquid level position of the suction head.

In some embodiments, the calculating the liquid level position candidate points of the upper and lower contours respectively according to the gray level first order difference sequence includes:

and taking the negative peak in the gray first-order difference sequence as a liquid level position candidate point of the upper contour and the lower contour.

In some embodiments, the method further comprises:

and if the point of the negative peak sequence with the first-order difference value smaller than the third threshold value cannot be searched in the gray first-order difference sequence, judging that no sample is absorbed.

In some embodiments, said determining whether a valid defect exists in said edge detection map based on said tip profile comprises:

removing the suction head contour from the edge detection image to obtain a defect contour image;

calculating the area of each contour in the defect contour map, and if the contour with the contour area not smaller than a fifth threshold exists in the defect contour map, judging that an effective defect exists in the defect contour map;

and if the contour with the contour area smaller than the fifth threshold exists in the defect contour map, judging the contour with the contour area smaller than the fifth threshold as an invalid defect, and after removing the invalid defect, judging that the sample suction is qualified if the contour with the contour area not smaller than the fifth threshold does not exist.

In some embodiments, removing the tip profile from the edge detection map to obtain a defect profile map comprises:

in the edge detection map, adjacent pixel values within a fourth threshold range of the suction head profile are set to be 0, and a defect profile map is obtained.

In some embodiments, the determining the quality of the sample suction according to the characteristics of the effective defects and the position of the liquid level of the sucker comprises:

marking the valid defects;

if the center of the effective defect is outside the contour of the suction head or is at a position other than a position between the liquid level position of the suction head and the top point of the suction head, judging that the sample is hung on the wall;

calculating the defect characteristics of the effective defects if the centers of the effective defects are in the inner part of the outline of the suction head or in the position between the liquid level position of the suction head and the top point of the suction head;

and if the roundness of the defect outline corresponding to the defect characteristic is larger than the fifth threshold, judging the defect outline to be a bubble, otherwise, judging the defect outline to be a sample wall hanging.

In some embodiments, the primary binarization of the tip image to obtain an effective region map includes:

and carrying out filtering processing on the sucker image.

In some embodiments, the primary binarization of the tip image to obtain an effective region map includes:

carrying out primary binarization on the suction head image to obtain a binary image;

carrying out pixel summation on the binary image in the horizontal direction and the vertical direction respectively to obtain a horizontal projection sequence and a vertical projection sequence;

respectively searching a starting position and a terminal position which are not zero in the horizontal projection sequence and the vertical projection sequence to be used as the boundary of the effective area of the sucker image;

and obtaining the effective area map according to the boundary of the effective area of the sucker image.

In a second aspect, an embodiment of the present invention provides a tip image processing apparatus, including:

the acquisition module is used for acquiring a sucker image;

the binarization module is used for carrying out primary binarization on the suction head image to obtain an effective area map;

the gradient detection module is used for carrying out gradient edge detection on the effective region map to obtain an edge detection map, and carrying out gradient binarization processing on the edge detection map to obtain a gradient binarization map;

the characteristic extraction module is used for extracting the characteristics of the suction head from the effective area map to obtain the characteristics of the suction head when the suction head is determined to exist in the gradient binary image, wherein the characteristics of the suction head comprise the contour of the suction head and the liquid level position of the suction head;

the defect judging module is used for judging whether effective defects exist in the edge detection image according to the suction head profile;

and the sample suction quality judging module is used for judging the sample suction quality according to the characteristics of the effective defects and the liquid level position of the suction head if the effective defects exist.

In a third aspect, an embodiment of the present invention provides a tip image processing apparatus, including:

at least one processor, and

a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method described above.

In a fourth aspect, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a tip image processing apparatus, cause the tip image processing apparatus to perform the method as described above.

According to the sucker image processing method, the sucker image processing device and the sucker image processing equipment, after a sucker image is obtained, primary binarization is carried out on the sucker image, and an effective area image with a sucker is positioned; performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image; when the gradient binary image is used for determining that the suction head exists, performing suction head feature extraction on the effective area image to obtain suction head features comprising the contour of the suction head and the liquid level position of the suction head; then, judging whether effective defects exist in the edge detection image according to the profile of the suction head, so that whether the sample suction condition corresponding to the suction head image is qualified can be judged; if the effective defect exists, the sample suction quality can be judged according to the characteristics of the effective defect and the liquid level position of the suction head, so that the sample suction quality can be accurately judged.

Drawings

One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.

FIG. 1 is a schematic view of a scene of an embodiment of a tip image processing method of the present invention;

FIG. 2 is a schematic flow chart diagram illustrating one embodiment of a tip image processing method according to the present invention;

FIG. 3 is a raw view of a tip image of one embodiment of a tip image processing method of the present invention;

FIG. 4 is a filtered image of a pipette tip image according to an embodiment of the method for processing a pipette tip image of the present invention;

FIG. 5 is a binarized image of a filtered image according to an embodiment of the method for processing a suction head image;

FIG. 6 is a view of an effective area of one embodiment of the image processing method of the suction head of the present invention;

FIG. 7 is a gradient edge detection map of one embodiment of a tip image processing method of the present invention;

FIG. 8 is a gradient binarization image of an embodiment of the tip image processing method of the present invention;

FIG. 9 is an edge detection view of one embodiment of a tip image processing method of the present invention;

FIG. 10 is a defect diagram of one embodiment of the image processing method for the suction head of the present invention;

FIG. 11 is a defect localization diagram of one embodiment of a tip image processing method of the present invention;

FIG. 12 is a schematic structural view of an embodiment of a tip image processing apparatus according to the present invention;

FIG. 13 is a schematic configuration diagram of an embodiment of a tip image processing apparatus according to the present invention;

fig. 14 is a schematic diagram showing the hardware configuration of a controller in one embodiment of the image processing apparatus for a pipette tip of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The sucker image processing method and the sucker image processing device provided by the embodiment of the invention can be applied to sucker image processing equipment.

It is understood that, in the tip image processing apparatus, a controller is provided as a main control center, and the suction condition inside the tip is determined, and the suction quality is determined.

And, in this suction head image processing equipment, still be equipped with the camera, carry out suction head image acquisition, as shown in fig. 1, the camera sets up in the side of the application of sample subassembly suction head position of automatic branch cup system for shoot the image of the application of sample subassembly suction head position of automatic branch cup system, and the opposite side of application of sample subassembly suction head position is provided with the backlight, in order to satisfy the environment when camera shoots application of sample subassembly suction head position.

The automatic cup separating system can realize functions of uncovering a disease sampling tube, sucking and transferring a sample, closing the cover of the virus sampling tube, sterilizing, filtering and the like, and utilizes the camera to carry out real-time image acquisition on a suction head part of a sample adding assembly of the automatic cup separating system, can carry out real-time analysis on a sample sucking condition, judges whether the sample sucking quantity is accurate or not, judges whether a sample hanging liquid exists on the outer wall of the suction head or not, and judges whether bubbles are sucked or not, thereby laying a foundation for the accuracy of subsequent PCR amplification and DNA analysis.

Referring to fig. 2, fig. 2 is a schematic flow chart of a tip image processing method according to an embodiment of the present invention, which can be executed by the controller 13 in the tip image processing apparatus, as shown in fig. 1, the method includes:

101: a tip image is acquired.

Specifically, a suction head image can be obtained through the camera as shown in fig. 1, and the suction head image is an image corresponding to a suction head part of the sample adding assembly of the automatic cup separating system.

And after the image of the sucker is collected by the camera, as shown in fig. 3, the original image of the sucker is collected and preprocessed.

In some embodiments, since there is noise in the image generally, in order to reduce the influence of the noise, the method may further include:

and carrying out filtering processing on the sucker image.

More specifically, when the tip image is subjected to filtering processing, a processing mode combining median filtering and mean filtering may be adopted. After a suction head image is obtained, a median filtering mode is used for removing gray abnormal jumping points, and the adopted median filtering formula 1 is as follows:

wherein f1(x, y) represents the output gray scale value at the (x, y) point in the tip image; s_xyA set of coordinates representing a rectangular sub-image window of size m x n centered at point (x, y); mean is an operation of taking a median of a group of data; g1(s, t) represents the input gray value at the point of the image (s, t) before filtering.

After obtaining the output gray-scale value f1(x, y), in order to further reduce the influence of noise on subsequent operations, the output gray-scale value f1(x, y) is subjected to an average filtering process, where the average filtering is as follows in formula 2:

wherein f1(s, t) is the median filtered Image, which is used as the input of the mean filtering, and the filtered Image Blur _ Image is obtained after the filtering processing, as shown in fig. 4, the original sucker Image is compared with the sucker Image after the filtering processing, and the filtering processing can obviously inhibit the Image noise.

102: and carrying out primary binarization on the suction head image to obtain an effective area image.

After obtaining the filtered suction head image, performing primary binarization on the filtered suction head image to obtain a binary image, as shown in fig. 5, and then determining an effective area map to be analyzed in the binary image, which may specifically include:

carrying out pixel summation on the binary image in the horizontal direction and the vertical direction respectively to obtain a horizontal projection sequence and a vertical projection sequence;

respectively searching a starting position and a terminal position which are not zero in the horizontal projection sequence and the vertical projection sequence to be used as the boundary of the effective area of the sucker image;

and obtaining the effective area map according to the boundary of the effective area of the sucker image.

Specifically, pixel summation is performed on the binary image in the horizontal direction and the vertical direction respectively to obtain a horizontal projection sequence and a vertical projection sequence, where the calculation mode of the horizontal projection sequence is formula 3, and the calculation mode of the vertical projection sequence is formula 4:

wherein, Binary _ Image is an Image after binarization; binary _ Image (i, j) is the pixel gray value of the ith row and the jth column of the binarized Image; width is the image width and height is the image height; i is an integer between 1 and the image width; j is an integer between 1 and the image Height.

Then, the starting position and the end position with the numerical value not being zero in the Horizontal _ array sequence and the Vertical _ array sequence are respectively searched as the boundary of the effective area of the sucker image. From the filtered Image, the effective region to be analyzed is intercepted, and an effective region map ROI _ Image is obtained, as shown in fig. 6.

103: and carrying out gradient edge detection on the effective region map to obtain an edge detection map, and carrying out gradient binarization processing on the edge detection map to obtain a gradient binarization map.

The gradient binary image can be used for judging whether the suction head exists or not and for positioning the root node of the suction head.

The edge detection map is then used to detect defects.

Specifically, gradient edge detection may be performed on the effective region map ROI _ Image to be analyzed by using a scharr operator. The Gradient maps in the horizontal direction and the vertical direction are respectively calculated to obtain a horizontal Gradient result and a vertical Gradient result, and then the horizontal Gradient result and the vertical Gradient result are subjected to Image fusion according to a preset weight (for example, a weight of 0.5), that is, the two Gradient maps are superimposed to obtain a complete Gradient edge detection map Gradient _ Image, as shown in fig. 7.

The scharr operator can be used for acquiring more gradient details of the image, and the suction head root node can be found in the subsequent operation. If the Canny algorithm is directly adopted for edge detection, the information of the root node position can be eliminated, so that the root node cannot be positioned, and the upper contour and the lower contour of the suction head cannot be further searched.

After the Gradient edge detection map Gradient _ Image shown in fig. 7 is obtained, since a lot of ring noise exists in the Gradient binary map Gradient _ Image, it is necessary to perform Image morphology processing, that is, to perform an opening operation processing on the Gradient edge detection map Gradient _ Image to reduce interference caused by the parasitic lines, so as to obtain a morphological Image morphh _ Image with the parasitic lines removed.

After the morphological Image Morph _ Image with the stray lines removed is obtained, the morphological Image Morph _ Image is subjected to binarization processing to obtain a Binary morphological Image Binary _ Image2, and the Binary morphological Image Binary _ Image2 is used as the gradient Binary Image. As shown in FIG. 8, the image after binarization retains valid data of the tip portion.

And 104, when the existence of the suction head is determined in the gradient binary image, performing suction head feature extraction on the effective area image to obtain suction head features, wherein the suction head features comprise a suction head outline and a suction head liquid level position.

Because the sample suction condition in the suction head is detected, whether the suction head exists in the gradient binary image needs to be detected, in some embodiments, the method may further include:

if the sum of the pixels of the gradient binary image is larger than a first threshold value, judging that a suction head exists in the gradient binary image;

and if the sum of the pixels of the gradient binary image is not greater than the first threshold value, judging that no suction head exists in the gradient binary image.

Specifically, the sum of all pixels in a gradient Binary Image (Binary morphology Image2) is calculated, and if the sum of the pixels is greater than a first threshold value Th1, a sucker is judged to be present in the gradient Binary Image; if the pixel sum is not larger than the first threshold Th1, it is determined that no suction head exists in the gradient binary image, and a prompt of no suction head is given. The sum of pixels is equation 5:

whether the suction head exists in the suction head image is judged by utilizing the gradient binary image, so that the accuracy of suction head detection is effectively improved.

In some embodiments, performing tip feature extraction on the active area map to obtain tip features, where the tip features include a tip profile and a tip liquid level position, and the tip feature extraction may include:

determining the search range of the suction head root node in the gradient binary image, and searching the suction head root node in the effective area image;

searching a local gray minimum value in the effective area graph by taking the suction head root node as a starting point and in the direction of the suction head vertex and taking the pixel range of a second threshold value where the two side outlines are located to obtain the two side outlines of the suction head;

determining the top point of the suction head according to the profiles of the two sides of the suction head, determining the profile of the suction head according to the profiles of the two sides of the suction head and the top point of the suction head, and determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic that gray level mutation exists at the junction of liquid and air.

Specifically, after obtaining the gradient Binary Image (Binary morphology Image Binary _ Image2) as shown in fig. 8, determining a search range of a tip root node in the Binary morphology Image Binary _ Image2, and during search, determining a search range of a root node in the effective region map in a column where the tip root position of the gradient Binary Image (Binary morphology Image Binary _ Image2) is located, that is, the last column; and finding a suction head root node in the search range of the effective area graph by using the characteristic of the gray local minimum of the root node.

Furthermore, in the row of the root position of the suction head, the first non-zero pixel position P _ upper is searched from top to bottom, the first non-zero pixel position P _ lower is searched from bottom to top, and if the midpoint of the P _ upper and the P _ lower is P _ mid, the search range of the upper root node is [ P _ upper, P _ mid ], and the search range of the lower root node is [ P _ mid, P _ lower ]. And searching for the Root node Root _ Upper and the Root node Root _ Lower of the suction head in the search range according to the characteristic that the contour is the local minimum of the gray scale, thereby obtaining the suction head Root node comprising the Root node Root _ Upper and the Root node Root _ Lower of the suction head.

After finding the Root node Root _ Upper and the Root node Root _ Lower on the suction head, respectively taking the Root node Root _ Upper and the Root node Root _ Lower on the suction head as starting points, and searching the next contour point of the suction head in the direction of the suction head. The determined search range is: and (3) respectively expanding the range of second threshold Th2 pixels to the upper and lower sides in the vertical direction by taking the position of the contour point as a reference, and longitudinally searching the next local gray minimum value point, namely the contour point, until the point within the ROI _ Image width is traversed. The calculation method of the contour point sequence is as follows:

boundary _ upper [ i ] ═ min (ROI _ Image (i, j-Th2),.., ROI _ Image (i, j + Th2)) formula 6;

boundary _ lower [ i ] ═ min (ROI _ Image (i, j-Th2),. > ROI _ Image (i, j + Th2)) formula 7;

wherein min is the operation of taking the minimum value in the data sequence, Boundary _ upper is the upper contour of the suction head, and Boundary _ lower is the lower contour of the suction head; i is an integer from 1 to the ROI _ Image width; j is an integer ranging from 1 to the ROI _ Image height on the ordinate of the last contour point.

And obtaining the upper outline Boundary _ upper of the suction head and the lower outline Boundary _ lower of the suction head as the two-side outlines of the suction head.

Then, determining the Top point of the sucker according to the two side profiles of the sucker, and determining the intersection point of the two side profiles as the Top point of the sucker, namely the intersection point between the upper profile Boundary _ upper of the sucker and the lower profile Boundary _ lower of the sucker as the Top point Root _ Top of the sucker.

And determining the contour of the suction head according to the contours of the two sides of the suction head and the Top point of the suction head, namely enclosing an upper contour Boundary _ upper, a lower contour Boundary _ lower and the Top point Root _ Top of the suction head to form the contour of the suction head.

And simultaneously, determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic of gray level jump at the junction of the liquid and the air. Specifically, in some embodiments, determining the position of the liquid level of the suction head according to the two-side profile and the characteristic of the presence of a gray-scale jump at the boundary between the liquid and the air may include:

calculating the gray level first-order difference of the gray level sequences of the upper contour and the lower contour to obtain a gray level first-order difference sequence;

respectively calculating liquid level position candidate points of the upper contour and the lower contour according to the gray level first-order difference sequence;

if first-order differential values meeting preset conditions are found in the gray first-order differential sequences of the upper contour and the lower contour respectively, judging that the sample is sucked, wherein the preset conditions are that the first-order differential values in the gray first-order differential sequences are smaller than the point of a negative peak of a third threshold value, and the corresponding position distance of the negative peak of the upper contour or the lower contour is not more than 2;

and averaging the positions of the negative peak points of the upper contour and the lower contour in the gray first-order difference meeting the preset condition to obtain the liquid level position of the suction head.

When determining the liquid level position of the suction head, firstly, calculating the gray level first-order difference of the gray level sequence of the upper contour and the gray level first-order difference sequence of the lower contour to obtain a gray level first-order difference sequence, namely respectively calculating the gray level first-order difference sequence of the upper contour Boundary _ upper of the suction head and the gray level first-order difference sequence of the lower contour Boundary _ lower of the suction head, wherein the specific formula 8 and the formula 9 are as follows:

gray delta _ upper [ i ] ═ Gray _ array _ upper [ i +1] -Gray _ array _ upper [ i ] equation 8;

gray delta _ lower [ i ] ═ Gray _ array _ lower [ i +1] -Gray _ array _ lower [ i ] equation 9;

wherein, GrayDelta _ upper is a gray level first-order difference sequence of an upper contour Boundary _ upper, and GrayDelta _ lower is a first-order difference sequence of a lower contour Boundary _ lower; the value of i ranges from 1 to an integer between-1 and the contour length of the upper contour Boundary _ upper or the contour length of the lower contour Boundary _ lower.

After the gray level first-order difference sequence of the upper outline Boundary _ upper of the suction head and the gray level first-order difference sequence of the lower outline Boundary _ lower of the suction head are obtained, calculating liquid level position candidate points of the upper outline and the lower outline according to the gray level first-order difference sequence. When the sample sucking assembly sucks a sample, the sample firstly reaches the top of the suction head and then flows to the root of the suction head, the gray value of a part containing liquid is larger, and the gray value of a part not containing liquid is smaller, as shown in fig. 6, therefore, the liquid level position generates an obvious gray increase from the root of the suction head to the top, a negative number with a larger absolute value is generated in the corresponding gray first-order difference sequence, namely, a negative peak is generated at the position of the liquid level by the gray first-order difference of the profile, and the negative peak in the gray first-order difference is taken as a candidate point of the liquid level position. Specifically, positions where the value of the gray-scale first-order difference sequence gray delta _ upper of the upper contour Boundary _ upper and the first-order difference sequence gray delta _ lower of the lower contour Boundary _ lower is smaller than the third threshold Th3 are searched for and can be used as liquid level position candidate points.

After the liquid level position candidate point is obtained, the determination of the liquid level position may be performed. Due to the influence of liquid hanging or air bubbles, a plurality of negative peaks with the gray first-order difference smaller than the third threshold value may exist. Assuming that m and n negative peaks whose grays are smaller than the third threshold Th3 are found in the gray level first order difference sequence gray delta _ upper of the upper contour Boundary _ upper and the first order difference sequence gray delta _ lower of the lower contour Boundary _ lower, respectively, then based on the symmetry of the upper contour and the lower contour, the liquid level position cannot exceed 2 between the two liquid level position candidate point positions of the upper contour and the lower contour, i.e. calculated by using the following formula 10:

equation 10 of | deltaPos _ upper [ i ] -deltaPos _ lower [ j ] | 2

The deltaPos _ upper is a negative spike sequence of which the upper profile gray scale is smaller than a third threshold Th 3; deltaPos _ lower is a negative spike sequence with lower profile gray less than a third threshold Th 3; i is an integer from 1 to m, and j is an integer from 1 to n.

If the point of the negative peak sequence with the first-order difference value of the gray first-order difference smaller than the third threshold value is not found, that is, the negative peak with the first-order difference value of the upper contour smaller than the third threshold value Th3 and the negative peak with the first-order difference value of the lower contour smaller than the third threshold value Th3 are not found, it is determined that no sample is sucked.

Otherwise, if the negative peak sequence point with the first-order difference value smaller than the third threshold value is found in the first-order difference of the gray scale, that is, the negative peak sequence point with the upper profile first-order difference smaller than the third threshold value Th3 and the negative peak sequence point with the lower profile first-order difference smaller than the third threshold value Th3 are found, and the corresponding position distance of the upper and lower profile negative peak points is not more than 2, so that the preset condition is met, and the sample is judged to be sucked.

And after determining that the sample is sucked, averaging the positions of the upper and lower profile negative peak points of which the gray first-order difference meets a preset condition to obtain the liquid level position of the suction head, namely outputting the positions of the upper and lower profile points meeting the requirement to be averaged to be used as the liquid level position of the suction head.

Through the gradient binarization mode, the sample suction quality of the cup separating system can be preliminarily judged, and the condition that the sample is not loaded on the suction head or the sample is not sucked by the suction head can be detected.

And 105, judging whether effective defects exist in the edge detection image according to the suction head profile.

After the suction head contour is determined, whether effective defects exist in the edge detection image or not is judged, and the effective defects can be bubbles or liquid hanging. As can be seen from the gradient edge detection graph in fig. 7, due to the problems of light source distribution and camera acquisition, annular noise is formed, and the periphery of the effective region map ROI _ Image is more obvious, so after the feature points and the tip contour of the tip are determined, more precise tip parts need to be cut out from the effective region map ROI _ Image to eliminate the influence of the annular noise.

When the annular noise is eliminated, the top point of the sucker is taken as a starting point in the horizontal direction, the root node is taken as an end point, and a certain width is outwards expanded on the basis of the upper root node and the lower root node in the vertical direction respectively to obtain an accurately positioned sucker Image Pip _ Image.

In some embodiments, determining whether a valid defect exists in the edge detection map based on the tip profile may include:

removing the suction head contour from the edge detection image to obtain a defect contour image;

calculating the area of each contour in the defect contour map, and if the contour with the contour area not smaller than a fifth threshold exists in the defect contour map, judging that an effective defect exists in the defect contour map;

and if the contour with the contour area smaller than the fifth threshold exists in the defect contour map, judging the contour with the contour area smaller than the fifth threshold as an invalid defect, and after removing the invalid defect, judging that the sample suction is qualified if the contour with the contour area not smaller than the fifth threshold does not exist.

Specifically, after the effective area map is accurately positioned, a suction head Image Pip _ Image is obtained, and edge detection is performed on the suction head Image Pip _ Image to obtain an edge detection map. The edge detection may be performed by Canny edge detection, which results in the Canny edge detection Image Canny _ Image, to locate the tip edge, as shown in figure 9.

Then, the tip profile is removed from the edge detection map, so as to obtain a Defect profile map, that is, in the edge detection map, the value of an adjacent pixel within a fourth threshold range of the tip profile is set to 0, so as to obtain a Defect profile map, specifically, in the edge detection map Canny _ Image, the value of an adjacent pixel within a fourth threshold Th4 along the upper profile Boundary _ upper and the lower profile Boundary _ lower of the tip profile is set to 0 in the vertical direction respectively in the upper and lower directions with the profile point as the center, so that the remaining non-zero pixels are the Defect profile, so as to obtain a Defect profile map Defect _ Image, as shown in fig. 10.

After obtaining the defect profile map, defect localization can be performed. Because the Defect outline Image includes some smaller outlines, the area of each outline in the Defect outline Image is calculated, and if the outline area in the Defect outline Image is not smaller than the outline with the fifth threshold value, the Defect outline Image is judged to have effective defects; and if the contour with the contour area smaller than the fifth threshold exists in the defect contour map, judging the contour with the contour area smaller than the fifth threshold as an invalid defect, and after removing the invalid defect, judging that the sample suction is qualified if the contour with the contour area not smaller than the fifth threshold does not exist. Namely, the contour with the area smaller than the fifth threshold Th5 is regarded as an interference contour, the interference contour can be removed, and the remaining contours are all effective defects; the contour whose area is larger than the fifth threshold Th5 is regarded as a valid defect finally detected.

106: and if the effective defects exist, judging the sample sucking quality according to the characteristics of the effective defects and the liquid level position of the sucking head.

In some embodiments, determining the quality of the sample according to the characteristics of the effective defect and the position of the liquid level of the sucker comprises:

marking the valid defects;

if the center of the effective defect is outside the contour of the suction head or is at a position other than a position between the liquid level position of the suction head and the top point of the suction head, judging that the sample is hung on the wall;

calculating the defect characteristics of the effective defects if the centers of the effective defects are in the inner part of the outline of the suction head or in the position between the liquid level position of the suction head and the top point of the suction head;

and if the roundness of the defect outline corresponding to the defect characteristic is larger than the fifth threshold, judging the defect outline to be a bubble, otherwise, judging the defect outline to be a sample wall hanging.

Specifically, after the finally detected effective defect is obtained, the type of the effective defect is determined. The minimum bounding rectangle of the valid defect can be obtained by using a function minAreaRect in OpenCV, and as shown in fig. 11, the valid defect finally detected is marked by a minimum bounding rectangle frame.

And then, the center of the minimum circumscribed rectangular frame is obtained as the center of the effective defect, and if the center of the effective defect is outside the outline of the suction head or is at a position other than the position between the liquid level position of the suction head and the top point of the suction head, the sample is judged to be hung on the wall. If the center of the effective defect is in the inner part of the sucker outline or the position between the liquid level position of the sucker and the top point of the sucker, further analysis is needed, for example, when the wall-hung water drop is on one surface of the sucker close to the light source, the defect can not be directly judged as a bubble because the defect is below the liquid level and in the sucker outline, at the moment, the defect characteristic needs to be further calculated so as to judge whether the bubble or the liquid is hung, and the roundness can be used as the defect characteristic to calculate the defect characteristic of the effective defect; and if the roundness of the defect outline corresponding to the defect characteristic is larger than the fifth threshold, judging the defect outline to be a bubble, otherwise, judging the defect outline to be a sample wall hanging. The calculation of roundness as a defect feature is shown in equation 11:

where Afa is the circularity of the contour, S is the area of the contour, and C is the perimeter of the contour.

And when the roundness of the defect contour is larger than a fifth threshold Th5, judging the defect contour to be a bubble, otherwise, judging the defect contour to be a wall hanging.

After the effective defects are positioned, the defect types can be judged by using the characteristics of the defects, the sample suction amount is prevented from being inaccurate due to the conditions that the liquid is hung on the suction head, the suction head sucks bubbles and the like, and the accuracy of subsequent nucleic acid detection is ensured.

According to the embodiment of the application, after a suction head image is obtained, primary binarization is carried out on the suction head image, and an effective area image with a suction head is positioned; performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image; when the gradient binary image is used for determining that the suction head exists, performing suction head feature extraction on the effective area image to obtain suction head features comprising the contour of the suction head and the liquid level position of the suction head; then, judging whether effective defects exist in the edge detection image according to the profile of the suction head, so that whether the sample suction condition corresponding to the suction head image is qualified can be judged; if the effective defect exists, the sample suction quality can be judged according to the characteristics of the effective defect and the liquid level position of the suction head, so that the sample suction quality can be accurately judged.

Accordingly, as shown in fig. 12, an embodiment of the present invention further provides a tip image processing apparatus, which may be used for a tip image processing device, where the tip image processing apparatus 600 includes:

an acquisition module 601, configured to acquire a tip image;

a binarization module 602, configured to perform primary binarization on the suction head image to obtain an effective area map;

a gradient detection module 603, configured to perform gradient edge detection on the effective region map to obtain an edge detection map, and perform gradient binarization processing on the edge detection map to obtain a gradient binarization map;

a feature extraction module 604, configured to perform, when it is determined that a suction head exists in the gradient binarization image, suction head feature extraction on the effective area image to obtain a suction head feature, where the suction head feature includes a suction head profile and a suction head liquid level position;

a defect determining module 605, configured to determine whether there is an effective defect in the edge detection map according to the suction head profile;

and a sample sucking quality judging module 606, configured to judge, if the effective defect exists, a sample sucking quality according to a characteristic of the effective defect and the position of the liquid level of the suction head.

According to the embodiment of the invention, after a suction head image is obtained, primary binarization is carried out on the suction head image, and an effective area image with a suction head is positioned; performing gradient edge detection on the effective region image to obtain an edge detection image, and performing gradient binarization processing on the edge detection image to obtain a gradient binarization image; when the gradient binary image is used for determining that the suction head exists, performing suction head feature extraction on the effective area image to obtain suction head features comprising the contour of the suction head and the liquid level position of the suction head; then, judging whether effective defects exist in the edge detection image according to the profile of the suction head, so that whether the sample suction condition corresponding to the suction head image is qualified can be judged; if the effective defect exists, the sample suction quality can be judged according to the characteristics of the effective defect and the liquid level position of the suction head, so that the sample suction quality can be accurately judged.

In other embodiments, as shown in fig. 13, the tip image processing device 600 includes: a suction head judging module 607 for:

if the sum of the pixels of the gradient binary image is larger than a first threshold value, judging that a suction head exists in the gradient binary image;

and if the sum of the pixels of the gradient binary image is not greater than the first threshold value, judging that no suction head exists in the gradient binary image.

In other embodiments, the feature extraction module 604 is further configured to:

determining the search range of the suction head root node in the gradient binary image, and searching the suction head root node in the effective area image;

searching a local gray minimum value in the effective area graph by taking the suction head root node as a starting point and in the direction of the suction head vertex and taking the pixel range of a second threshold value where the two side outlines are located to obtain the two side outlines of the suction head;

determining the top point of the suction head according to the profiles of the two sides of the suction head, determining the profile of the suction head according to the profiles of the two sides of the suction head and the top point of the suction head, and determining the liquid level position of the suction head according to the profiles of the two sides of the suction head and the characteristic that gray level mutation exists at the junction of liquid and air.

In some embodiments, the feature extraction module 604 is further configured to:

determining the search range of root nodes in the effective area map at the row of the suction head root position of the gradient binary map;

and finding a suction head root node in the search range in the effective area graph by using the characteristic of the local minimum value of the gray level of the root node.

In some embodiments, the feature extraction module 604 is further configured to:

and determining the intersection point of the two side profiles of the suction head as the top point of the suction head.

In some of these embodiments, the two-sided profile includes an upper profile and a lower profile; the feature extraction module 604 is further configured to:

calculating the gray level first-order difference of the gray level sequences of the upper contour and the lower contour to obtain a gray level first-order difference sequence;

respectively calculating liquid level position candidate points of the upper contour and the lower contour according to the gray level first-order difference sequence;

if first-order differential values meeting preset conditions are found in the gray first-order differential sequences of the upper contour and the lower contour respectively, judging that the sample is sucked, wherein the preset conditions are that the first-order differential values in the gray first-order differential sequences are smaller than the point of a negative peak of a third threshold value, and the corresponding position distance of the negative peak of the upper contour or the lower contour is not more than 2;

and averaging the positions of the negative peak points of the upper contour and the lower contour in the gray first-order difference meeting the preset condition to obtain the liquid level position of the suction head.

In some embodiments, the feature extraction module 604 is further configured to:

and taking the negative peak in the gray first-order difference sequence as a liquid level position candidate point of the upper contour and the lower contour.

In some embodiments, as shown in fig. 13, the apparatus 600 further comprises a sample suction determining module 608 for:

and if the point of the negative peak sequence with the first-order difference value smaller than the third threshold value cannot be searched in the gray first-order difference sequence, judging that no sample is absorbed.

In some embodiments, the defect determining module 605 is further configured to:

removing the suction head contour from the edge detection image to obtain a defect contour image;

and calculating the area of each contour in the defect contour map, and if the contour with the contour area not smaller than a fifth threshold exists in the defect contour map, judging that the defect contour map has effective defects.

In some embodiments, the apparatus 600 further comprises a sample qualifying with suction module 609 to:

and if the contour with the contour area smaller than the fifth threshold exists in the defect contour map, judging the contour with the contour area smaller than the fifth threshold as an invalid defect, and after removing the invalid defect, judging that the sample suction is qualified if the contour with the contour area not smaller than the fifth threshold does not exist.

In some embodiments, the sample quality determination module 606 is further configured to:

in the edge detection map, adjacent pixel values within a fourth threshold range of the suction head profile are set to be 0, and a defect profile map is obtained.

In some embodiments, the defect determining module 605 is further configured to:

marking the valid defects;

if the center of the effective defect is outside the contour of the suction head or is at a position other than a position between the liquid level position of the suction head and the top point of the suction head, judging that the sample is hung on the wall;

calculating the defect characteristics of the effective defects if the centers of the effective defects are in the inner part of the outline of the suction head or in the position between the liquid level position of the suction head and the top point of the suction head;

and if the roundness of the defect outline corresponding to the defect characteristic is larger than the fifth threshold, judging the defect outline to be a bubble, otherwise, judging the defect outline to be a sample wall hanging.

In some embodiments, the apparatus 600 further comprises a filtering module 610 configured to:

and carrying out filtering processing on the sucker image.

In some embodiments, the binarization module 602 is further configured to:

carrying out primary binarization on the suction head image to obtain a binary image;

carrying out pixel summation on the binary image in the horizontal direction and the vertical direction respectively to obtain a horizontal projection sequence and a vertical projection sequence;

respectively searching a starting position and a terminal position which are not zero in the horizontal projection sequence and the vertical projection sequence to be used as the boundary of the effective area of the sucker image;

and obtaining the effective area map according to the boundary of the effective area of the sucker image.

It should be noted that the above-mentioned apparatus can execute the method provided by the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. For technical details which are not described in detail in the device embodiments, reference is made to the methods provided in the embodiments of the present application.

Fig. 14 is a schematic diagram of a hardware configuration of a controller in an embodiment of the tip image processing apparatus, and as shown in fig. 14, the controller 13 includes:

one or more processors 131, memory 132. Fig. 14 illustrates an example of one processor 131 and one memory 132.

The processor 131 and the memory 132 may be connected by a bus or other means, and fig. 14 illustrates the connection by the bus as an example.

The memory 132 is a non-volatile computer-readable storage medium, and can be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the tip image processing method in the embodiment of the present application (for example, the acquiring module 601, the binarizing module 602, the gradient detecting module 603, the feature extracting module 604, the defect determining module 605, the sample sucking quality determining module 606, the tip determining module 607, the sample sucking determining module 608, the sample sucking qualified module 609, and the filtering module 610 shown in fig. 12-13). The processor 131 executes various functional applications of the controller and data processing, that is, implements the tip image processing method of the above-described method embodiment, by executing nonvolatile software programs, instructions, and modules stored in the memory 132.

The memory 132 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the tip image processing apparatus, and the like. Further, the memory 132 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 132 may optionally include memory located remotely from the processor 131, which may be connected to a tip image processing device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory 132 and, when executed by the one or more processors 131, perform the tip image processing method of any of the method embodiments described above, e.g., performing method steps 101-106 of FIG. 2 described above; the functions of the modules 601 and 606 in fig. 12 and the modules 601 and 610 in fig. 13 are realized.

The product can execute the method provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the methods provided in the embodiments of the present application.

Embodiments of the present application provide a non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by one or more processors, such as one of the processors 131 of fig. 14, enable the one or more processors to perform the tip image processing method of any of the above-described method embodiments, such as performing the above-described method steps 101-106 of fig. 2; the functions of the modules 601 and 606 in fig. 12 and the modules 601 and 610 in fig. 13 are realized.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a general hardware platform, and may also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.