阅读说明：本技术 一种基于摄像头的平面图案识别方法及其装置与应用 (Planar pattern recognition method based on camera, device and application thereof ) 是由 赖亚明 刘新 于 2019-09-27 设计创作，主要内容包括：一种基于摄像头的平面图案识别方法及其装置与应用,包括选择所需平面图案,预先将全部平面图案制成高斯蒙版模型：识别时摄像头对其下方的平面图案进行拍摄；利用纹理或连通域算法将平面图案从拍到的图像中抠出；将抠出后的图案转成矩形,使图像调正并调整为与蒙版模型中相一致的尺寸,然后采用自动色阶处理；再进行高斯模糊；最后以前述高斯蒙版模型进行识别。其可用于对图书封面或内页进行识别。其装置包括光源、摄像头、单片机和存储芯片,存储芯片用来预存蒙版模型；摄像头进行拍摄之后,单片机以前述方法进行识别。本发明可操作性强,识别准确,具有良好的扩展性能,能够扩展出位置识别和动作识别等功能,可以应用于点读机等电子阅读设备。(A planar pattern recognition method based on camera, its device and application, including choosing the required planar pattern, making all the planar patterns into Gauss mask model in advance: during identification, the camera shoots a plane pattern below the camera; utilizing texture or connected domain algorithm to extract the plane pattern from the photographed image; converting the stripped pattern into a rectangle, aligning and adjusting the image to be the same as the size in the mask model, and then adopting automatic color gradation processing; then carrying out Gaussian blur; and finally, identifying by the Gaussian mask model. Which can be used to identify the book cover or inner page. The device comprises a light source, a camera, a singlechip and a storage chip, wherein the storage chip is used for prestoring a mask model; after the camera shoots, the singlechip recognizes by the method. The invention has strong operability, accurate identification and good expansion performance, can expand the functions of position identification, action identification and the like, and can be applied to electronic reading equipment such as a point reading machine and the like.)

1. A plane pattern recognition method based on a camera comprises the following steps:

step 1, selecting a required plane pattern, and preparing all plane patterns into a Gaussian mask model in advance:

scanning or shooting the plane pattern into a digital image, and normalizing the sizes of all the digital images into an MxN size;

the specific method comprises the following steps:

1.1) decomposing the digital image into CMYK four colors or RGB-Gray four colors;

1.2) carrying out Gaussian blur on each channel, wherein the blur radius is not less than 2;

1.3) respectively superposing the information of four channels of all the digital images, wherein the information of the four channels corresponding to the ith digital image is negatively superposed, and the rest are positively superposed, so that mask models containing four channels and the number of the mask models is equal to the number of pages;

1.4) then, carrying out automatic color gradation processing on each mask model to enable the numerical value to fill the interval of 0-255;

step 2, during identification, firstly, shooting a plane pattern below the camera by the camera; the cameras are usually positioned right above, left above and front above the plane pattern during shooting;

step 3, utilizing texture or connected domain algorithm to extract the plane pattern from the shot image; under the two conditions that the camera is positioned at the upper left and the upper front, the extracted plane pattern is trapezoidal;

step 4, if the extracted planar pattern is not rectangular, performing inverse perspective transformation on the non-rectangular image to transform the non-rectangular image into a rectangle;

step 5, performing rotation operation on the rectangular image obtained in the step 4 to align the image so as to accord with the visual angle of a reader and facilitate the subsequent step processing; when the camera is positioned at the upper left part, the camera rotates 90 degrees clockwise; when the camera is positioned at the front upper part, the camera is rotated by 180 degrees, and when the camera is positioned at the right upper part, the camera is not rotated;

step 6, adjusting the shape of the image in the previous step into a rectangle with the size of M multiplied by N pixels, then adopting automatic color gradation processing to ensure that the original gray value of the image is uniformly interpolated to an interval of 0 ~ 255, and then carrying out Gaussian blur with the blur radius not less than 2;

and 7, identifying the Gaussian blurred image obtained in the step 6 by using the Gaussian mask model generated in the step 1 to obtain an identification result, wherein the identification result is as follows:

and (3) completely superposing the image obtained in the step (6) and the Gaussian mask model obtained in the step (1), and selecting the template number with the minimum standard deviation after superposition as an output result, thereby identifying the type of the planar pattern.

2. The method as claimed in claim 1, wherein the step 7 further comprises setting a threshold value, threshold value field [0,1], and if the calculated standard deviation after superposition is not lower than the preset threshold value, determining that the planar pattern to be measured cannot match any of the mask models in the step 1.

3. Use of the method of claim 1, characterized in that the method is used for identifying book covers.

4. Use of the method as claimed in claim 1, characterized in that the method is used for identifying pages in a book.

5. A plane pattern recognition device based on a camera is characterized by comprising a light source, a camera (1), a single chip microcomputer and a storage chip, wherein the photographing direction of the camera (1) is consistent with the illumination direction of the light source; the memory chip is used for prestoring mask models of various planar patterns obtained in the step 1; after the plane pattern is shot by the cameras (1) positioned right above, left above and front above the plane pattern, the single chip microcomputer processes the shot plane pattern in a step 2-6 mode, and identifies an image obtained after processing in a step 7 mode.

6. A plane pattern recognition device based on a camera comprises a desk lamp and is characterized by further comprising a camera 1, a single chip microcomputer and a storage chip, wherein the single chip microcomputer and the storage chip are arranged in the desk lamp, and the photographing direction of the camera (1) is consistent with the illumination direction of an illuminating lamp; and the storage chip in the desk lamp is used for prestoring mask models of various planar patterns obtained in the mode of the step 1; after the plane pattern is shot by the cameras (1) positioned right above, left above and front above the plane pattern, the single chip microcomputer in the desk lamp processes the shot plane pattern in a step 2-6 mode, and identifies an image obtained after processing in a step 7 mode.

Technical Field

The invention relates to a plane pattern recognition method, in particular to a plane pattern recognition method based on a camera, a device and application thereof, and belongs to the field of machine vision and intelligent electronics.

Background

With the development of computer technology, devices such as high-speed cameras and scanners have gradually become equipped with various intelligent processing functions. But its intelligent functions are limited to image correction of the target object and preset target object recognition capabilities. For example, the high-speed scanner can correct perspective deformation caused by bending of a paper surface; a scanner with the ability to identify objects can automatically identify photographs and text from images, including the photographs and text into different types of formatted documents, respectively.

The existing device has the defects that the capacity of identifying the preset target is limited, and the preset target cannot be identified according to the target type defined by a user. The angle of photographing or scanning is regulated, for example, a high-speed camera must take orthogonal photographing at an angle vertically directed to the paper, and the condition of large-angle oblique photographing cannot be tolerated.

In addition, the algorithm for setting the target object identification template by the existing device is complex, the identification template needs to be manufactured by methods such as pattern recognition and the like, and a common user cannot master the identification template, so that the identification template is usually built in product hardware after being designed by a manufacturer, and the application expansion capability of a product is limited.

On the other hand, a point reading machine is a common learning and education product. The existing point reading machine product is greatly improved compared with the initial stage, but is still limited to the improvement of content production level and the enhancement of multimedia effect, and the technical basis of 'specially produced books' or 'specially produced drawings' is not changed in the aspect of basic user operation. A disadvantage of the prior art solutions is that the reading must be a specially produced reading. The existing point-reading machine (pen) usually adopts a special bar code or a special optical symbol embedded in printed matter to realize the indexing of the content pointed by the user. Production of the reading material and copyright problems cause additional costs. The most critical steps of the point reading machine still include the identification of the front cover of the reading material and the identification of the content of the reading material, however, the point reading machine in the prior art still stays on the traditional identification mode, and the pattern identification technology is hardly adopted.

Disclosure of Invention

The invention aims to provide a plane pattern recognition method based on a camera, which can be divided into three conditions that the camera is positioned right above the plane pattern, the camera is positioned at the left upper part of the plane pattern and the camera is positioned at the front upper part of the plane pattern according to the relative position of the camera and the plane pattern, so that three methods which are consistent in the main flow and different in details are formed.

It is another object of the present invention to provide the use of the above method.

It is still another object of the present invention to provide a camera-based flat pattern recognition apparatus, which can be classified into a general case apparatus and a desk lamp type flat pattern recognition apparatus combined with a desk lamp.

The method comprises the following specific steps:

a plane pattern recognition method based on a camera comprises the following steps:

step 1, selecting a required plane pattern, and preparing all plane patterns into a Gaussian mask model in advance:

scanning or shooting the plane pattern into a digital image, and normalizing the sizes of all the digital images into an MxN size;

the specific method comprises the following steps:

1.1) decomposing the digital image into CMYK four colors or RGB-Gray four colors;

1.2) carrying out Gaussian blur on each channel, wherein the blur radius is not less than 2;

1.3) respectively superposing the information of four channels of all the digital images, wherein the information of the four channels corresponding to the ith digital image is negatively superposed, and the rest are positively superposed, so that mask models containing four channels and the number of the mask models is equal to the number of pages;

1.4) then, carrying out automatic color gradation processing on each mask model to enable the numerical value to fill the interval of 0-255;

step 2, during identification, firstly, shooting a plane pattern below the camera by the camera; the cameras are usually positioned right above, left above and front above the plane pattern during shooting;

step 3, utilizing texture or connected domain algorithm to extract the plane pattern from the shot image; under the two conditions that the camera is positioned at the upper left and the upper front, the extracted plane pattern is trapezoidal;

step 4, if the extracted planar pattern is not rectangular, performing inverse perspective transformation on the non-rectangular image to transform the non-rectangular image into a rectangle;

step 5, performing rotation operation on the rectangular image obtained in the step 4 to align the image so as to accord with the visual angle of a reader and facilitate the subsequent step processing; when the camera is positioned at the upper left part, the camera rotates 90 degrees clockwise; when the camera is positioned at the front upper part, the camera is rotated by 180 degrees, and when the camera is positioned at the right upper part, the camera is not rotated;

step 6, adjusting the shape of the image in the previous step into a rectangle with the size of M multiplied by N pixels, then adopting automatic color gradation processing to ensure that the original gray value of the image is uniformly interpolated to an interval of 0 ~ 255, and then carrying out Gaussian blur with the blur radius not less than 2;

and 7, identifying the Gaussian blurred image obtained in the step 6 by using the Gaussian mask model generated in the step 1 to obtain an identification result, wherein the identification result is as follows:

and (3) completely superposing the image obtained in the step (6) and the Gaussian mask model obtained in the step (1), and selecting the template number with the minimum standard deviation after superposition as an output result, thereby identifying the type of the planar pattern.

The step 7 further comprises setting a threshold value, namely a threshold value range [0,1], and if the calculated standard deviation after superposition is not lower than a preset threshold value, judging that the to-be-detected planar pattern cannot be matched with any mask model in the step 1.

The method is applied to the identification of book covers.

The method is applied to the identification of the book inner pages.

A plane pattern recognition device based on a camera is characterized by comprising a light source, the camera, a single chip microcomputer and a storage chip, wherein the photographing direction of the camera is consistent with the illumination direction of the light source; the memory chip is used for prestoring mask models of various planar patterns obtained according to the step mode; and after the plane pattern is shot by the cameras positioned right above, left above and front above the plane pattern, the single chip microcomputer processes the shot plane pattern in a step 2-6 mode, and identifies an image obtained after processing in a step 7 mode.

A planar pattern recognition device based on a camera comprises a desk lamp and is characterized by further comprising the camera, a single chip microcomputer and a storage chip, wherein the single chip microcomputer and the storage chip are arranged in the desk lamp, and the photographing direction of the camera 1 is consistent with the illumination direction of an illuminating lamp; and the storage chip in the desk lamp is used for prestoring mask models of various planar patterns obtained in the mode of the step 1; after the planar pattern is shot by the cameras 1 positioned right above, left above and front above the planar pattern, the single chip microcomputer in the desk lamp processes the shot planar pattern in a step 2-6 mode, and identifies an image obtained after processing in a step 7 mode.

Advantages of the invention

The invention provides a specific method for manufacturing the identification template, and the method has strong operability. The method disclosed by the invention is accurate in identification and high in efficiency, and can be used for quickly and accurately identifying the plane pattern recorded into the database.

The invention has good expansion performance, and can develop various electronic reading devices on the basis. When the image recognition device provided by the invention is matched with a pen-shaped input device for use, the functions of position recognition, action recognition and the like can be expanded, and the use scenes of the invention are further enriched.

The method can be applied to point-reading machine equipment, can improve the interaction process between a user and a reading material, particularly between teenagers and teaching materials, integrates the point-reading machine and the desk lamp, enables the user to perform point-reading operation in a posture closer to daily reading, and is beneficial to cultivating the teenagers to form good sitting postures and reading habits.

Drawings

FIG. 1 is an image of the Gray channel of the four reading covers.

Fig. 2 is a gaussian blur processed image of Gray channel images of the four reading covers in fig. 1.

Fig. 3 shows the result of superimposing four-channel information of two patterns according to the present invention.

Fig. 4 is a Gray channel mask model of the four patterns shown in fig. 1.

Fig. 5 is a schematic view of the camera positioned at the top left of the planar pattern (spine towards camera).

Fig. 6 is an image of a planar pattern in the field of view of the camera in the situation of fig. 5.

Fig. 7 is a schematic view of the camera positioned at the upper left of the plane pattern, as viewed from the side (upper edge of the book towards the camera).

Fig. 8 is an image of a planar pattern in the field of view of the camera in the situation of fig. 7.

Fig. 9 is an image obtained by photographing with the camera positioned at an arbitrary angle above the plane pattern.

Fig. 10 is a schematic drawing of the matting plan pattern of fig. 9.

Fig. 11 is a schematic diagram of an inverse perspective transformation.

Fig. 12 is a diagram showing the actual effect of the inverse perspective transformation.

Fig. 13 is a diagram showing an actual effect of inverse perspective transformation of a trapezoidal pattern.

FIG. 14 is a schematic view of the apparatus of the present invention.

Wherein, 1, camera, 2, base, 3, support arm, 4, light.

Detailed Description

A plane pattern recognition method based on a camera comprises the following steps:

step 1, selecting a required plane pattern, and preparing all plane patterns into a Gaussian mask model in advance:

scanning or shooting the planar pattern into a digital image, and normalizing the sizes of all the digital images into M multiplied by N, wherein the values of M and N are not less than 100 in the experiment of the invention;

if the planar pattern type is large, the values of M and N need to be increased, but in general, the value of M × N does not need to exceed 100,000, and the higher the value, the better the value is not.

The specific method comprises the following steps:

1.1) decomposing the digital image into CMYK four colors or RGB-Gray four colors;

it is noted that K or Gray are not completely independent color components, but are associated with CMY or RGB. For example, for RGB-Gray, the Gray value is equal to about R × 0.299 + G × 0.587 + B × 0.114. But the K or Gray channel contains the fusion information of the other three channels, which is the most important channel in most cases; and taking all N books into CMYK channels or RGB-Gray and unifying the sizes by adopting a multi-channel fusion mode. The plane pattern is taken as a reading cover, the channel is taken as Gray, and Gray images of the four reading covers are shown in figure 1.

The images of the Gray channel reading of the reading cover of FIG. 1 are clearly different. Wherein, the first is the left-right light and shade difference; the second sheet is the difference between the upper and lower light and shade; the third is uneven light and shade distribution; the fourth sheet is relatively uniform in brightness distribution.

1.2) carrying out Gaussian blur on each channel, wherein the blur radius is not less than 2; in the experiment, 3 pixels are used as the fuzzy radius; as shown in fig. 2.

1.3) respectively superposing the information of four channels of all the digital images, wherein the information of the four channels corresponding to the ith digital image is negatively superposed, and the rest are positively superposed, so that mask models containing four channels and the number of the mask models is equal to the number of pages;

1.4) then, carrying out automatic color gradation processing on each mask model to enable the numerical value to fill the interval of 0-255; the stacking process is shown in figure 3 of the drawings,

fig. 4 is an example of superimposing the Gray channels (the Gray channel is only one of the four R-G-B-Gray channels). That is, the template of each reading was composed of 4 templates (R template, G template, B template, Gray template, or C template, M template, Y template, K template). For convenience of description, only 1 type will be taken as an example.

Step 2, during identification, firstly, shooting a plane pattern below the camera by the camera; the cameras are usually positioned right above, left above and front above the plane pattern during shooting; the perspective image of the plane pattern can be obtained under the latter two conditions;

the situation where the camera is located at the upper left is shown in fig. 5, and the image in the field of view of the camera is shown in fig. 6.

To facilitate subsequent matting operations, a planar pattern can be placed on a table top having a single base color. The placement of the planar pattern can also be made to meet the requirement that the optical axis of the camera can penetrate through the center of the planar pattern, so that the parameter setting is reduced in the implementation of the algorithm.

Of course, there may be a case where the camera is positioned above the plane pattern and performs photographing at an arbitrary angle to the plane pattern, and the photographed image is as shown in fig. 9.

Fig. 5 is a schematic view of the camera positioned at the top left of the planar pattern (spine towards camera).

Fig. 6 is an image of a planar pattern in the field of view of the camera in the situation of fig. 5.

The situation with the camera at the upper front is shown in fig. 7, and the image in the field of view of the camera is shown in fig. 8.

Fig. 7 is a schematic view of the camera positioned at the upper left of the plane pattern, as viewed from the side (upper edge of the book towards the camera).

Fig. 8 is an image of a planar pattern in the field of view of the camera in the situation of fig. 7.

Fig. 9 is an image obtained by photographing with the camera positioned at an arbitrary angle above the plane pattern.

Step 3, utilizing texture or connected domain algorithm to extract the plane pattern from the shot image; under the two conditions that the camera is positioned at the upper left and the upper front, the extracted plane pattern is trapezoidal; as previously shown in fig. 6, 8; in the latter case, the extracted pattern is a parallelogram, as shown in fig. 10.

Step 4, if the extracted planar pattern is not rectangular, performing inverse perspective transformation on the non-rectangular image to transform the non-rectangular image into a rectangle; the effect of the reverse perspective transformation to the prior art is shown in fig. 11, 12, and 13;

in the specific operation, the parameter of perspective deformation is based on 'whether to convert into rectangle', if the parameter can not be converted into rectangle after one conversion, the parameter can be adjusted by adopting successive approximation method, and multiple conversions are executed until the converted image is rectangle.

Step 5, performing rotation operation on the rectangular image obtained in the step 4 to align the image so as to accord with the visual angle of a reader and facilitate the subsequent step processing; when the camera is positioned at the upper left part, the camera rotates 90 degrees clockwise; when the camera is positioned at the front upper part, the camera is rotated by 180 degrees, and when the camera is positioned at the right upper part, the camera is not rotated.

Therefore, the method of the invention can be divided into three specific methods according to the relative positions of the camera and the plane pattern, and the method is different from the method in that whether reverse perspective transformation is needed after the matting and how to perform rotation and alignment operations.

Step 6, adjusting the shape of the image in the previous step into a rectangle with the size of M multiplied by N pixels, then adopting automatic color gradation processing to ensure that the original gray value of the image is uniformly interpolated to an interval of 0 ~ 255, and then carrying out Gaussian blur with the blur radius not less than 2;

the automatic tone scale process is a prior art, such as a range of gray scale values of 0 ~ 255, but the range of gray scale in the image may be only 50 ~ 200 (that is, there are no pure white and pure black pixels in the image), in which case the automatic tone scale function uniformly interpolates the range of 50 ~ 200 to 0 ~ 255, so as to maximize the contrast of the whole image.

And 7, identifying the Gaussian blurred image obtained in the step 6 by using the Gaussian mask model generated in the step 1 to obtain an identification result, wherein the identification result is as follows:

and (3) completely superposing the image obtained in the step (6) and the Gaussian mask model obtained in the step (1), and selecting the template number with the minimum standard deviation after superposition as an output result, thereby identifying the type of the planar pattern.

Further, the step 7 further includes setting a threshold value, wherein the threshold value is [0,1], and if the calculated standard deviation after superposition is not lower than a preset threshold value, it is determined that the planar pattern to be detected cannot be matched with any mask model in the step 1.

The method is applied to the book cover identification.

The method is applied to the identification of the book inner pages.

A plane pattern recognition device based on a camera is characterized by comprising a light source, a camera 1, a single chip microcomputer and a storage chip, wherein the photographing direction of the camera 1 is consistent with the illumination direction of the light source; the memory chip is used for prestoring mask models of various planar patterns obtained in the step 1; after the planar pattern is shot by the cameras 1 positioned right above, left above and front above the planar pattern, the single chip microcomputer processes the shot planar pattern in a step 2-6 mode, and identifies an image obtained after processing in a step 7 mode.