阅读说明：本技术 面向无约束条件下的车牌定位及识别方法、系统 (License plate positioning and identifying method and system under unconstrained condition ) 是由 许亮 曹玉社 李峰 于 2020-03-26 设计创作，主要内容包括：本发明提供了一种面向无约束条件下的车牌定位及识别方法,对无约束条件下的车辆图像样本进行数据增强处理；将增强样本数据输入至目标检测网络,进行车牌预定位；矫正车牌预定位结果,得到样本的准确车牌定位结果；将定位结果送至车牌识别网络,采用搜索算法进行车牌后过滤识别；最终得到训练后的车牌定位及识别模型；在实际应用部署中,基于训练得到的车牌定位及识别模型,进行无约束条件下的车牌定位和车牌识别。同时提供了一种用于执行上述方法的车牌定位及识别系统。本发明实用性强,可以满足大部分场景下车牌定位和车牌识别的准确率；涉及算法的复杂度低,可集成性强,运行速度快,可以有效保证实时性。(The invention provides a license plate positioning and identifying method under an unconstrained condition, which is used for carrying out data enhancement processing on a vehicle image sample under the unconstrained condition; inputting the enhanced sample data into a target detection network, and performing license plate prepositioning; correcting the license plate pre-positioning result to obtain an accurate license plate positioning result of the sample; sending the positioning result to a license plate recognition network, and filtering and recognizing license plates by adopting a search algorithm; finally obtaining a trained license plate positioning and identifying model; in practical application deployment, license plate positioning and license plate recognition under an unconstrained condition are carried out on the basis of a license plate positioning and recognition model obtained through training. Meanwhile, a license plate positioning and identifying system for executing the method is provided. The method has strong practicability, and can meet the accuracy of license plate positioning and license plate identification in most scenes; the algorithm is low in complexity, strong in integratability and high in running speed, and real-time performance can be effectively guaranteed.)

1. A license plate positioning and identifying method oriented to an unconstrained condition is characterized by comprising the following steps:

s1, obtaining a vehicle image sample under an unconstrained condition;

s2, performing data enhancement processing on the vehicle image sample acquired in S1 to form an enhanced sample training data set;

s3, randomly inputting vehicle image sample data in the training sample data set obtained in the S2 into a target detection network, and performing license plate pre-positioning on the sample data;

s4, correcting the license plate pre-positioning result obtained in the S3 to obtain an accurate and complete license plate picture;

s5, sending the license plate picture obtained in the S4 to a license plate recognition network for license plate number recognition, sequentially matching a plurality of number sequences obtained by recognition with a predefined template set, and returning a first number sequence which is successfully matched;

s6, after the target detection network in S3 and the license plate recognition network in S5 are trained, an end-to-end license plate positioning and license plate recognition model under an unconstrained condition is obtained;

s7, deploying the license plate positioning and recognition model obtained in the S6 under an unconstrained condition, obtaining a field vehicle image, and inputting the field vehicle image into the license plate positioning and recognition model;

s8, detecting the position of a complete license plate in an input image according to the license plate positioning part in the license plate positioning and license plate recognition model, and positioning the license plate;

and S9, according to the license plate positioning and the license plate recognition part in the license plate recognition model, carrying out license plate number recognition on the positioned license plate image to obtain a corresponding license plate code sequence.

2. The unconstrained condition-oriented license plate positioning and recognition method of claim 1, wherein in S1, vehicle images collected from different scenes are added to the vehicle image sample based on the existing image sample; wherein the different scenarios include: different lighting, different angular deflections of the vehicle relative to the camera on different axes, and blurred scenes.

3. The unconstrained condition-oriented license plate positioning and recognition method according to claim 1, wherein in S2, the data enhancement process includes: performing geometric enhancement and/or appearance enhancement on the vehicle image sample; the resulting vehicle image data is made to contain a raw data portion and an enhanced data portion.

4. The unconstrained condition-oriented license plate positioning and recognition method according to claim 3, wherein the geometric enhancement comprises: rotating and/or randomly cropping; wherein:

the rotation is used for adapting the trained model to different placing positions of the vehicle in the input image; in the rotating process, the corresponding ground truth coordinate is correspondingly transformed; filling a black area around the rotated sample data by using average pixels of the sample;

the random cutting enables the license plate model to adapt to the condition that the vehicle in the input image is incomplete;

the appearance enhancement includes: motion blur, gaussian blur and/or Gamma transformation; wherein:

the motion blur and the Gaussian blur are used for adapting the license plate model to the image blur phenomenon in the input image;

the Gamma transformation enhances the contrast and brightness of sample data, and is used for adapting the license plate model to the illumination difference in the actual scene.

5. The unconstrained condition-oriented license plate positioning and recognition method of claim 1, wherein in S3, the target detection network is a yolov3-tiny network, and a loss function of the yolov3-tiny network is:

where W, H are the width and height of the feature map, A is the prior frame number, λ_class,λ_coord,λ_obj,λ_noobjRespectively representing the weight coefficients of the terms in loss,representing whether the divided mesh contains a target, g_tRepresenting true calibration value, b_ijkIn order to be a predictive value for the network,is the intersection ratio of the prediction frame and the real frame;

in formula (1):

first itemIs a classification error;

second itemIs the position error of the prediction frame;

item IIIA confidence error for a prediction box containing the target;

the fourth term λ_noobj·1_IOU＜thresh·(0-b_ijk)²The confidence error for a prediction box that does not contain a target.

6. The unconstrained condition-oriented license plate positioning and recognition method according to claim 1, wherein in S4, the correcting the license plate prepositioning result includes:

-regressive coordinates for obtaining a license plate image comprising a complete license plate;

-a license plate correction unit for obtaining a license plate image with a front side in a horizontal direction.

7. The license plate positioning and recognition method under the unconstrained condition of claim 6, wherein the regression coordinates adopt a regression loss method, specifically, a weight and a bias parameter of a target detection network are fixed, a full connection layer and a regression loss layer are sequentially connected at the rear end of the target detection network, and the weight and the bias parameter of the full connection layer and the regression loss layer are trained and updated according to license plate position information in sample data; and/or

The license plate correction adopts a trained STN network.

8. The unconstrained-condition-oriented license plate positioning and recognition method of claim 1, wherein in S5, the license plate recognition network employs a L PRNet license plate recognition network, and the L PRNet license plate recognition network employs a Beam search algorithm to perform post-filtering recognition on license plate numbers to obtain multiple most likely number sequences.

9. A license plate positioning and recognition system oriented to unconstrained conditions is characterized by comprising:

-a training sample acquisition module: the training sample acquisition module acquires vehicle images of a vehicle in different scenes as training samples;

-a training sample enhancement module: the training sample enhancement module performs data enhancement processing on a training sample to form an enhanced sample training data set;

-a target detection network module for license plate pre-positioning of sample data in a training sample data set;

the correction module corrects the pre-positioned license plate image to finally obtain normalized license plate sample data after license plate positioning;

the license plate recognition network module is used for carrying out license plate recognition on the sample data subjected to license plate positioning to obtain a license plate sequence matched with the predefined template set.

10. The unconstrained condition-oriented license plate locating and recognizing system according to claim 9, further comprising any one or more of:

-the target detection network module employs yolov3-tiny network;

-the rectification module comprises a regression coordinate unit and a rectification license plate unit; wherein:

the regression coordinate unit obtains a license plate image completely containing a license plate;

the license plate correcting unit is used for obtaining a license plate image with the front surface in the horizontal direction;

-the regression coordinate unit employs a full connection layer and a regression loss layer connected to the back end of the target detection network;

-the orthotic license plate unit employs a trained STN network;

-the license plate recognition network module employs an L PRNet license plate recognition network.

Technical Field

The invention relates to the technical field of image processing and application, in particular to a license plate positioning and identifying method and system under an unconstrained condition.

Background

In the application process of a real scene, the accuracy and the real-time performance of license plate positioning and recognition are very important. The license plate positioning and recognition under the unconstrained condition require that the algorithm can position the complete license plate position in the image under the condition of small error, and simultaneously recognize characters in the positioned license plate accurately and in real time.

At present, the license plate positioning and identification mainly adopt the following methods:

(1) based on the situation that the camera is over against the license plate, a sample containing the vehicle is shot, and the license plate position in the sample is positioned by adopting the following three modes:

① processing the sample by traditional pattern recognition algorithm, such as binarization and morphology processing, to obtain processed characteristic image, and locating the license plate position in the image by using prior information;

②, training a machine learning classification model (such as SVM) by using a license plate and a large number of non-license plate samples in advance, traversing all sub-regions in the image, and judging whether the image contains a license plate region by using the trained classification model;

③, sending the whole sample into a target detection CNN network, detecting whether the image contains a license plate, and positioning the position of the license plate.

(2) After the license plate position in the image is positioned, the character is divided, and the divided character is obtained by a projection method or the like.

(3) After the segmented character blocks are obtained, specific characters, letters and numbers represented by each character need to be recognized, and the commonly used method for recognizing the characters is as follows:

① matching the segmented characters with the characters in the database, and using the character with the highest matching degree as the recognized character;

② a character recognition network is trained in advance and used to recognize the segmented character blocks.

Although the above method achieves higher accuracy in some scenarios, the following drawbacks still exist:

(1) the accuracy rate is low under an unconstrained condition, and in an actual scene, due to the influence of different factors such as illumination (forward light, backward light, side light, illumination intensity and the like), weather (clear, cloudy, rainy and snowy), vehicle positions (front and different-axis offset at different angles) and the like, samples and algorithms of the training algorithm at the present stage face a great challenge;

(2) the current license plate algorithm is generally divided into three stages: the three stages of license plate positioning, character segmentation and license plate recognition are respectively operated to bring the following problems: on one hand, time is consumed, each step needs a certain time, and meanwhile, in some algorithms, vehicles of a given sample are detected firstly, and then the license plate is detected on the basis; on the other hand, in the accuracy, for example, in the character segmentation process, if there is aging of the license plate and slight shielding around the license plate, the character blocks obtained by using the projection method will be inaccurate.

(3) The types of automobiles running on the highway are not only common household cars, but also other unusual automobiles such as police cars and the like, the license plates of the automobiles are greatly different from those of the common automobiles, and most algorithms in the current stage cannot be deployed under the unconstrained condition due to the difference.

At present, no explanation or report of the similar technology of the invention is found, and similar data at home and abroad are not collected.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides the license plate positioning and identifying method and system which can be deployed under the unconstrained condition and can meet the requirements of real-time performance and accuracy and are oriented under the unconstrained condition. In the license plate positioning process, a yolov3-tiny network is adopted, and short operation time is ensured in the process of obtaining high accuracy. Meanwhile, the character segmentation and the license plate recognition processes are combined, and final end-to-end recognition is realized by using one network. After the license plate is positioned, in order to ensure the accuracy of the subsequent process, the positioned license plate is further subjected to normalization processing.

The invention is realized by the following technical scheme.

According to one aspect of the invention, a license plate positioning and identifying method oriented to an unconstrained condition is provided, which comprises the following steps:

s1, obtaining a vehicle image sample under an unconstrained condition;

s2, performing data enhancement processing on the vehicle image sample acquired in S1 to form an enhanced sample training data set;

s3, randomly inputting vehicle image sample data in the training sample data set obtained in the S2 into a target detection network, and performing license plate pre-positioning on the sample data;

s4, correcting the license plate pre-positioning result obtained in the S3 to obtain an accurate and complete license plate picture;

s5, sending the license plate picture obtained in the S4 to a license plate recognition network for license plate number recognition, sequentially matching a plurality of number sequences obtained by recognition with a predefined template set, and returning a first number sequence which is successfully matched;

s6, after the target detection network in S3 and the license plate recognition network in S5 are trained, an end-to-end license plate positioning and license plate recognition model under an unconstrained condition is obtained;

s7, deploying the license plate positioning and recognition model obtained in the S6 under an unconstrained condition, obtaining a field vehicle image from and to a vehicle, and inputting the field vehicle image into the license plate positioning and recognition model;

s8, detecting the position of the whole license plate in the input image according to the license plate positioning part in the license plate positioning and license plate recognition model, and positioning the license plate;

and S9, identifying the license plate number of the positioned license plate image according to the license plate positioning and the license plate identification part in the license plate identification model to obtain a corresponding license plate number code sequence.

Preferably, in S1, for the vehicle image sample, the vehicle images collected from different scenes are added on the basis of the existing image sample; wherein the different scenarios include: different lighting, different angular deflections of the vehicle relative to the camera on different axes, and blurred scenes.

Preferably, in S2, the data enhancement processing includes: performing geometric enhancement and/or appearance enhancement on the vehicle image sample; the resulting vehicle image data is made to contain a raw data portion and an enhanced data portion.

Preferably, the geometric enhancement comprises: rotating and/or randomly cropping; wherein:

the rotation is used for adapting the trained model to different placing positions of the vehicle in the input image; in the rotating process, the corresponding ground truth coordinate is correspondingly transformed; filling a black area around the rotated sample data by using average pixels of the sample;

the random cutting enables the license plate model to adapt to the condition that the vehicle in the input image is incomplete;

the appearance enhancement includes: motion blur, gaussian blur and/or Gamma transformation; wherein:

the motion blur and the Gaussian blur are used for adapting the license plate model to the image blur phenomenon in the input image;

the Gamma transformation enhances the contrast and brightness of sample data, and is used for adapting the license plate model to the illumination difference in the actual scene.

Preferably, in S3, the target detection network is a yolov3-tiny network, and the loss function of the yolov3-tiny network is:

where W, H are the width and height of the feature map, A is the prior frame number, λ_class,λ_coord,λ_obj,λ_noobjRespectively representing the weight coefficients of the terms in loss,representing whether the divided mesh contains a target, g_tRepresenting true calibration value, b_ijkIn order to be a predictive value for the network,is the intersection ratio of the prediction frame and the real frame;

in formula (1):

first itemIs a classification error;

second itemIs the position error of the prediction frame;

item IIIA confidence error for a prediction box containing the target;

the fourth term λ_noobj·1_IOU＜thresh·(0-b_ijk)²The confidence error for a prediction box that does not contain a target.

Preferably, in S4, the correcting the license plate prepositioning result includes:

-regressive coordinates for obtaining a license plate image comprising a complete license plate;

-a license plate correction unit for obtaining a license plate image with a front side in a horizontal direction.

Preferably, the regression coordinates adopt a regression loss method, specifically, the weights and the bias parameters of the target detection network are fixed, a full connection layer and a regression loss layer are sequentially connected to the rear end of the target detection network, and the weights and the bias parameters of the full connection layer and the regression loss layer are trained and updated according to license plate position information in sample data.

Preferably, the license plate correction adopts a trained STN network.

Preferably, in S5, the license plate recognition network employs a L PRNet license plate recognition network, and the L PRNet license plate recognition network employs a Beam search algorithm to perform post-filtering recognition on license plate numbers to obtain multiple most likely number sequences.

According to another aspect of the present invention, there is provided an unconstrained license plate location and identification system, comprising:

-a training sample acquisition module: the training sample acquisition module acquires vehicle images of a vehicle in different scenes as training samples;

-a training sample enhancement module: the training sample enhancement module performs data enhancement processing on a training sample to form an enhanced sample training data set;

-a target detection network module for license plate pre-positioning of sample data in a training sample data set;

the correction module corrects the license plate image subjected to the pre-positioning to obtain sample data subjected to license plate positioning;

the license plate recognition network module is used for carrying out license plate recognition on the sample data subjected to license plate positioning to obtain a license plate sequence matched with the predefined template set.

Preferably, the target detection network module adopts yolov3-tiny network.

Preferably, the correcting module comprises a regression coordinate unit and a correcting license plate unit; wherein:

the regression coordinate unit obtains a license plate image completely containing a license plate;

and the license plate correcting unit is used for obtaining a license plate image with the front surface in the horizontal direction.

Preferably, the regression coordinate unit adopts a full connection layer and a regression loss layer connected to the rear end of the target detection network.

Preferably, the license plate correcting unit adopts a trained STN network.

Preferably, the license plate recognition network module adopts an L PRNet license plate recognition network.

Compared with the prior art, the invention has the following beneficial effects:

1. the license plate locating and identifying method and system oriented to the unconstrained condition, provided by the invention, adopt yolov3-tiny network to pre-locate the license plate of the sample data, can realize real time and ensure the beneficial effect of precision.

2. The license plate positioning and identifying method and system under the unconstrained condition, provided by the invention, adopt an L PRNet license plate identification network, wherein the license plate identification is carried out by adopting post-filtering and Beam searching methods, and the beneficial effect of end-to-end license plate identification can be realized.

3. The license plate positioning and identifying method and system under the constraint-free condition are high in practicability and can meet the accuracy of license plate positioning and license plate identification in most scenes; meanwhile, the complexity of the related algorithm is low, the integratability is strong, the operation speed is high, and the real-time performance can be effectively ensured.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

fig. 1 is a schematic diagram of the operation and structure of a license plate location and identification method and system under an unconstrained condition according to a preferred embodiment of the present invention.

Detailed Description

The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

The embodiment of the invention provides a license plate positioning and identifying method under an unconstrained condition, which comprises the following steps:

s1, obtaining a vehicle image sample under an unconstrained condition;

s2, performing data enhancement processing on the vehicle image sample acquired in S1 to form an enhanced sample training data set;

s3, randomly inputting vehicle image sample data in the training sample data set obtained in the S2 into a target detection network, and performing license plate pre-positioning on the sample data;

s4, correcting the license plate pre-positioning result obtained in the S3 to obtain an accurate and complete license plate picture;

s5, sending the license plate picture obtained in the S4 to a license plate recognition network for license plate number recognition, sequentially matching a plurality of number sequences obtained by recognition with a predefined template set, and returning a first number sequence which is successfully matched;

s6, after the target detection network in S3 and the license plate recognition network in S5 are trained, an end-to-end license plate positioning and license plate recognition model under an unconstrained condition is obtained;

s7, deploying the license plate positioning and recognition model obtained in the S6 under an unconstrained condition, obtaining a field vehicle image from and to a vehicle, and inputting the field vehicle image into the license plate positioning and recognition model;

s8, detecting the position of a complete license plate in an input image according to the license plate positioning part in the license plate positioning and license plate recognition model, and positioning the license plate;

and S9, identifying the license plate number of the positioned license plate image according to the license plate positioning and the license plate identification part in the license plate identification model to obtain a corresponding license plate number code sequence.

The above-described solution is further described in detail below with reference to a preferred embodiment.

In the license plate positioning and identifying method oriented to the unconstrained condition, a yolov3-tiny network is adopted as a target detection network in the license plate positioning process, and short operation time is ensured in the process of obtaining high accuracy. Meanwhile, the character segmentation and the license plate recognition processes are combined, and a network is used for realizing final end-to-end recognition.

Specifically, the license plate positioning and identifying method oriented to the unconstrained condition comprises the following steps:

step S1, obtaining a vehicle image sample under an unconstrained condition;

step S2, performing data enhancement processing on the vehicle image sample acquired in the step S1 to form a training sample data set;

step S3, randomly inputting vehicle image sample data in the training sample data set obtained in the step S2 into the existing yolov3-tiny network with speed and accuracy, and pre-positioning license plates of the sample data;

step S4, correcting the license plate pre-positioning result obtained in the step S3 to obtain an accurate and complete license plate position;

and S5, sending the license plate picture obtained in the step S4 to a L PRNet license plate recognition network, performing post-filtering recognition on the license plate by adopting a Beam search algorithm to obtain a plurality of most probable number sequences, sequentially matching the plurality of number sequences obtained by recognition with a predefined template set, and returning a first sequence successfully matched with the predefined template set, wherein the predefined template set can be a Chinese vehicle license plate standard rule template set, and the first sequence is determined after being sequenced through a set sequencing criterion, for example, the first sequence can be a sequence with the highest score output by the network, namely, the output of the network is subjected to ascending sequencing, and the first sequence is returned.

Step S6, after training yolov3-tiny network in S3 and L PRNet license plate recognition network in S5, an end-to-end license plate positioning and license plate recognition model under the unconstrained condition can be obtained;

step S7, deploying the license plate positioning and recognition model under an unconstrained condition, acquiring a vehicle image, and inputting the vehicle image into the license plate positioning and recognition model;

step S8, detecting the position of a complete license plate in an input image according to the license plate positioning part in the license plate positioning and license plate recognition model, and positioning the license plate;

and step S9, according to the license plate positioning and the license plate recognition part in the license plate recognition model, performing license plate recognition on the vehicle image after the license plate positioning to obtain the corresponding license plate characters.

In the preferred embodiment, the target detection network adopts the yolov3-tiny network, which can take into account the accuracy and the running speed, i.e. compared with other networks, the yolov3-tiny network has high accuracy and low running speed, and the license plate recognition network adopts the L PRNet license plate recognition network, which can realize the end-to-end license plate recognition, i.e. compared with other networks, the network has strong deployment in the actual environment and low complexity.

The license plate location and identification method for the unconstrained condition provided by the embodiment of the invention is further described in detail with reference to fig. 1.

As shown in fig. 1, the license plate location and identification method under the unconstrained condition provided by the embodiment of the present invention includes two main processes: license plate location and license plate recognition are described below.

License plate positioning

The main purpose of the license plate positioning process is as follows: and giving an input image, and accurately detecting the position of the complete license plate in the image. In order to enable the license plate positioning method to be suitable for unconstrained scenes under actual conditions, the method comprises the following steps:

1. in the process of collecting training samples, on the basis of the existing samples, vehicle samples collected from different scenes are added as much as possible in the embodiment, and the scenes comprise: different lighting (including direct light, reverse light, side light, and different light intensities), different angle deflections of the vehicle relative to the camera on different axes (pitch, yaw, and yaw), blur, and the like;

2. in the training process, in order to make the license plate positioning method adapt to more scenes and expand the data set, the embodiment performs online data enhancement in the following way: the sample sent into the network comprises two parts of data, wherein one part of data is original data, the other part of data is online data enhancement data, and the types of data enhancement are as follows:

(1) geometric reinforcement: comprises the following steps:

rotation (and corresponding ground truth coordinates are also transformed accordingly), wherein the black area around the rotated image is filled with average pixels, this type of enhancement is to adapt the license plate location method to the different positions of the vehicles in the input image;

-random cropping, the enhancement of which simulates the situation in which the vehicle is not complete in the input image.

(2) Appearance enhancement, including the following:

motion blur and gaussian blur, mainly to adapt the license plate positioning method to image blur phenomena in the scene due to car bodies or other factors;

gamma transformation, image contrast and brightness enhancement, and the license plate positioning method is better adapted to illumination in an actual scene;

3. the network parameters of yolov3-tiny network, yolov3-tiny network are shown in table 1:

TABLE 1

Layer Type	Parameter
		#1 Input	416 × 3 RGB image
#2 Convolution+MaxPooling	3*3/1+2*2/2
		#3 Convolution+MaxPooling	3*3/1+2*2/2
#4 Convolution+MaxPooling	3*3/1+2*2/2
		#5 Convolution+MaxPooling	3*3/1+2*2/2
#6 Convolution+MaxPooling	3*3/1+2*2/2
		#7 Convolution+MaxPooling	3*3/1+2*2/1
#8 Convolution	3*3/1
		#9 Convolution	1*1/1
#10 Convolution	3*3/1
		#11 Convolution	1*1/1
#12#9+convolution+upsample	1*1/1
		#13 concat#6+#12
#14 Convolution	3*3/1
		#15 Convolution	1*1/1

The loss function is:

where W, H are the width and height of the feature map, A is the prior frame number, λ_class,λ_coord,λ_obj,λ_noobjRespectively representing the weight coefficients of the terms in loss,representing whether the divided mesh contains a target, g_tRepresenting true calibration value, b_ijkIn order to be a predictive value for the network,is the intersection ratio of the prediction box and the real box. First term in formula (1)For classification errors, second termTo predict the position error of the frame, a third termThe fourth term λ is the confidence error of the prediction box containing the target_noobj·1_IOU＜thresh·(0-b_ijk)²The confidence error for a prediction box that does not contain a target.

4. In an unconstrained scene, after the yolov3-tiny network obtains the license plate in the input image, the license plate needs to be further corrected, and the method specifically comprises the following two steps:

(1) and (5) returning coordinates, wherein the operation is to obtain a license plate image completely containing the license plate. Therefore, a simple regression loss layer is trained, specifically, the weight and the bias parameters of the yolov3-tiny network are fixed, a full connection layer and the regression loss layer are connected behind the network, and the weight and the bias parameters of the two layers are trained and updated according to the license plate position information in a training sample. And finally, obtaining accurate license plate position information.

(2) And (3) correcting the license plate, wherein the license plate with the front surface in the horizontal direction is obtained.

Second, license plate recognition

1. After the pre-processed license plate is obtained, the pre-processed license plate is sent to an L PRNet license plate recognition network, and the network parameters of the network are shown in the following tables 2 and 3.

TABLE 2L PR network architecture

TABLE 3 Small basic block network architecture

Layer Type	Parameters
		Input	Cin*H*W
Convolution	1*1/1
		Convolution	3*1/1/1
Convolution	1*3/1/1
		Convolution	1*1/1

(2) Meanwhile, post-filtering and Beam searching are adopted, wherein the post-filtering is used for obtaining the first N most probable sequences found by the Beam searching, and returning the first sequence matched with the predefined template set.

Third, testing stage

From the above training phase it can be derived: a license plate positioning and recognizing model; therefore, in the testing stage, namely in practical use, the vehicle images in most scenes can be captured and sent into the license plate positioning and identifying method, and the corresponding license plate characters can be output by the method.

Based on the license plate positioning and recognition method under the unconstrained condition provided by the embodiment of the invention, the embodiment of the invention also provides a license plate positioning and recognition system under the unconstrained condition, and the system can be used for executing the method.

The license plate positioning and identifying system facing to the unconstrained condition comprises:

-a training sample acquisition module: the training sample acquisition module acquires a training sample of a vehicle image;

-a training sample enhancement module: the training sample enhancement module performs data enhancement processing on a training sample to form an enhanced sample training data set;

a yolov3-tiny network module, wherein the yolov3-tiny network module performs license plate pre-positioning on sample data in a training sample data set;

the correction module corrects the license plate image subjected to the pre-positioning to obtain sample data subjected to license plate positioning;

and the L PRNet license plate recognition network module is used for carrying out license plate recognition on the sample data subjected to license plate positioning to obtain a license plate sequence matched with the predefined template set.

The license plate positioning and recognizing method and system oriented to the unconstrained condition, provided by the embodiment of the invention, have strong practicability, and can meet the accuracy of license plate positioning and license plate recognition in most scenes; meanwhile, the complexity of the related algorithm is low, the integratability is strong, the operation speed is high, and the real-time performance can be effectively ensured.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.