阅读说明：本技术 一种基于fpga的二值神经网络车牌识别方法及系统 (FPGA-based binary neural network license plate recognition method and system ) 是由 訾晶 金婕 张旭欣 付闯闯 王钰 陈美好 于 2019-09-29 设计创作，主要内容包括：本发明涉及基于FPGA的二值神经网络车牌识别方法及系统,方法包括：步骤1：利用图像预处理模块对输入图像做细化处理,并得到灰度图；步骤2：利用车牌定位提取模块对灰度图进一步处理以完成车牌的定位提取；步骤3：利用车牌字符分割模块对定位提取的车牌字符进行分割,并二值化形成尺寸固定的图像块；步骤4：对二值神经网络模块中的二值神经网络进行训练,利用训练完毕的二值神经网络模型对尺寸固定的图像块进行识别,并输出结果,方法配套的系统包括图像预处理模块、车牌定位提取模块、车牌字符分割模块和二值神经网络模块且基于FPGA平台实现。本发明将神经网络与FPGA硬件相结合,充分发挥两者优势,在保证车牌识别精度同时,实现了高效、低功耗。(The invention relates to a binary neural network license plate recognition method and system based on an FPGA (field programmable gate array), wherein the method comprises the following steps: step 1: an image preprocessing module is used for carrying out refinement processing on an input image and obtaining a gray-scale image; step 2: further processing the gray level image by using a license plate positioning and extracting module to complete positioning and extracting of a license plate; and step 3: utilizing a license plate character segmentation module to segment the license plate characters extracted by positioning, and binarizing to form an image block with a fixed size; and 4, step 4: the method comprises the steps of training a binary neural network in a binary neural network module, identifying image blocks with fixed sizes by using a trained binary neural network model, and outputting results. The invention combines the neural network and the FPGA hardware, fully exerts the advantages of the neural network and the FPGA hardware, and realizes high efficiency and low power consumption while ensuring the license plate recognition precision.)

1. A binary neural network license plate recognition method based on FPGA is characterized by comprising the following steps:

step 1: an image preprocessing module is used for carrying out refinement processing on an input image and obtaining a gray-scale image;

step 2: further processing the gray level image by using a license plate positioning and extracting module to complete positioning and extracting of a license plate;

and step 3: utilizing a license plate character segmentation module to segment the license plate characters extracted by positioning, and binarizing to form an image block with a fixed size;

and 4, step 4: and training a binary neural network in the binary neural network module, identifying the image block with the fixed size by using the trained binary neural network model, and outputting a result.

2. The FPGA-based binary neural network license plate recognition method of claim 1, wherein the step 1 specifically comprises: and utilizing an image preprocessing module to perform enhancement processing on the input image, adjusting the contrast and the brightness, correcting an inclined image caused by a shooting angle, and graying the image to obtain a grayscale image.

3. The FPGA-based binary neural network license plate recognition method of claim 1, wherein the step 2 specifically comprises: and (3) firstly, carrying out corrosion and expansion operations on the gray-scale image by using a license plate positioning extraction module to protrude the edge outline of the license plate, then searching the outline by using an edge detection algorithm, screening and removing interference options according to the outline area and the length-width ratio, and finally outputting a rectangular frame which has the minimum outline area and is in line with the length-width ratio of the license plate, namely correspondingly finishing the positioning extraction of the license plate.

4. The FPGA-based binary neural network license plate recognition method of claim 1, wherein the step 3 specifically comprises: and (3) performing character segmentation on the license plate characters extracted by positioning by using a license plate character segmentation module according to the size of a standard license plate and adopting a method of gradually shifting a central point to the right, and performing binarization to form an image block with a fixed size.

5. The FPGA-based binary neural network license plate recognition method of claim 4, wherein the sizes include 28x28 and 32x 32.

6. The FPGA-based binary neural network license plate recognition method of claim 1, wherein the binary excitation function used in the binary neural network in the step 4 is a sign function.

7. The FPGA-based binary neural network license plate recognition method of claim 1, characterized in that the binary neural network in step 4 is built by using an FPGA platform.

8. A system for the FPGA-based binary neural network license plate recognition method of any one of claims 1-7, the system comprising:

the image preprocessing module is used for carrying out enhancement, graying and denoising preprocessing on the acquired image;

the license plate positioning and extracting module is used for positioning, extracting and outputting the license plate in the preprocessed image;

the license plate character segmentation module is used for segmenting the license plate characters by adopting a method of gradually shifting the central point to the right, and binarizing to form an image block with a fixed size;

and the binary neural network module is used for generating an optimal network model through the training data, identifying the input image and finally outputting a result.

Technical Field

The invention relates to the technical field of machine learning, in particular to a binary neural network license plate recognition method and system based on an FPGA.

Background

With the rapid development of economy, vehicles are more and more, a large number of vehicles are arranged in residential districts or various public places, and the collection, the identification and the preservation of license plate information are more and more important for the convenience of management and the guarantee of personal property safety.

In addition, the traffic system is becoming intelligent, wherein the collection and identification of the license plate is an important part, and the application in the traffic monitoring system, the high-speed automatic charging system, the traffic flow detection system and other intelligent traffic systems is becoming more and more extensive.

In recent years, with the development of Artificial Intelligence (AI) technology, its application fields are more and more extensive, such as face recognition systems based on Convolutional Neural Network (CNN). However, the conventional neural network based on floating point operation has a large number of parameters and a large amount of computation, and occupies a lot of computing resources. On the other hand, when the license plate is identified, the deep learning algorithm based on the GPU (graphics processing unit) adopts a pure software manner for extracting the license plate outline and the character outline, which inevitably reduces the efficiency of processing a large amount of data.

Disclosure of Invention

The invention aims to overcome the defects in the prior art and provide a binary neural network license plate recognition method and system based on an FPGA.

The purpose of the invention can be realized by the following technical scheme:

a binary neural network license plate recognition method based on FPGA comprises the following steps:

step 1: an image preprocessing module is used for carrying out refinement processing on an input image and obtaining a gray-scale image;

step 2: further processing the gray level image by using a license plate positioning and extracting module to complete positioning and extracting of a license plate;

and step 3: utilizing a license plate character segmentation module to segment the license plate characters extracted by positioning, and binarizing to form an image block with a fixed size;

and 4, step 4: and training a binary neural network in the binary neural network module, identifying the image block with the fixed size by using the trained binary neural network model, and outputting a result.

Further, the step 1 specifically includes: and utilizing an image preprocessing module to perform enhancement processing on the input image, adjusting the contrast and the brightness, correcting an inclined image caused by a shooting angle, and graying the image to obtain a grayscale image.

Further, the step 2 specifically includes: and (3) firstly, carrying out corrosion and expansion operations on the gray-scale image by using a license plate positioning extraction module to protrude the edge outline of the license plate, then searching the outline by using an edge detection algorithm, screening and removing interference options according to the outline area and the length-width ratio, and finally outputting a rectangular frame which has the minimum outline area and is in line with the length-width ratio of the license plate, namely correspondingly finishing the positioning extraction of the license plate.

Further, the step 3 specifically includes: and (3) performing character segmentation on the license plate characters extracted by positioning by using a license plate character segmentation module according to the size of a standard license plate and adopting a method of gradually shifting a central point to the right, and performing binarization to form an image block with a fixed size.

Further, the sizes include 28 × 28 and 32 × 32.

Further, the binarization excitation function used by the binarization neural network in the step 4 is a sign function.

Further, the hardware establishment of the binary neural network in the step 4 adopts an FPGA platform.

The invention also provides a system for the FPGA-based binary neural network license plate recognition method, which comprises the following steps:

the image preprocessing module is used for carrying out enhancement, graying and denoising preprocessing on the acquired image;

the license plate positioning and extracting module is used for positioning, extracting and outputting the license plate in the preprocessed image;

the license plate character segmentation module is used for segmenting the license plate characters by adopting a method of gradually shifting the central point to the right, and binarizing to form an image block with a fixed size;

and the binary neural network module is used for generating an optimal network model through the training data, identifying the input image and finally outputting a result.

Compared with the prior art, the invention has the following advantages:

(1) according to the invention, based on an FPGA design system, firstly, the efficiency of license plate outline extraction is improved by the design of a hardware platform, secondly, the extraction of character outlines is abandoned, the character segmentation is directly carried out by adopting a method of gradually shifting a central point to the right, then, the calculation amount is greatly reduced by adding an FPGA-based binary neural network, and the operation efficiency is further improved.

(2) In the system, the input of the FPGA is collected by an external camera, and an image preprocessing module, a license plate positioning and extracting module, a license plate character segmentation module and a binary neural network module are designed in the FPGA. The image preprocessing module is mainly used for carrying out a series of processing such as enhancement, graying, denoising and the like on an acquired image, the license plate positioning extraction module is mainly used for positioning and extracting and outputting a license plate in the image, the license plate character segmentation module is mainly used for segmenting and binarizing license plate characters to form image blocks with fixed sizes and inputting the image blocks to the next layer, the binary neural network module is used for generating an optimal network model through training a large amount of data, identifying an input image and finally outputting a result, due to the fact that the neural network is introduced into a matching method, the generalization capability and the adaptability of the whole system are improved through supervised learning, in addition, the binary neural network can greatly reduce the calculated amount, the parallel execution advantage of FPGA is utilized, the efficiency is improved while the accuracy is guaranteed, and the power consumption is low.

Drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is a block diagram of the FPGA module design framework of the system of the present invention;

FIG. 3 is a flowchart illustrating the operation of the image pre-processing module of the system of the present invention;

FIG. 4 is a flowchart of the operation of the license plate location extraction module in the system of the present invention;

FIG. 5 is a flowchart of the operation of the license plate character segmentation module in the system of the present invention;

FIG. 6 is a flow chart of the operation of the binary neural network module in the system of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.