阅读说明：本技术 一种车牌字符定位方法及相关设备 (License plate character positioning method and related equipment ) 是由 古川南 周庆标 王�忠 于 2021-08-19 设计创作，主要内容包括：本申请公开了一种车牌字符定位方法及相关设备,包括：对待处理的车辆图像进行车牌粗定位,从车辆图像中截取车牌字符感兴趣区域,得到第一特征图像；通过训练后的车牌字符检测模型对第一特征图像进行处理,得到多个目标检测框,并从中确定车牌字符定位结果；其中,车牌字符检测模型被配置为,具备对输入的图像提取图像特征,并基于卷积层对图像特征进行卷积处理,得到第二特征图像,通过目标框检测层从第二特征图像中确定多个目标检测框的能力；其中,卷积层的卷积核大小以及所述目标框检测层的检测框的宽高比匹配于车牌的形状特性。本申请级联了车牌粗定位、车辆字符检测模型处理和模型输出的后处理,能够比较精确地实现车牌字符定位。(The application discloses a license plate character positioning method and related equipment, comprising the following steps: carrying out coarse license plate positioning on a vehicle image to be processed, and intercepting a license plate character region of interest from the vehicle image to obtain a first characteristic image; processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames, and determining a license plate character positioning result from the target detection frames; the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate. The method and the device have the advantages that the coarse license plate positioning, the vehicle character detection model processing and the model output post-processing are cascaded, and the license plate character positioning can be realized more accurately.)

1. A license plate character positioning method is characterized by comprising the following steps:

carrying out coarse license plate positioning on a vehicle image to be processed to obtain coarse license plate position information;

based on the rough license plate position information, intercepting a license plate character region of interest from the vehicle image to obtain a first characteristic image;

processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

determining a license plate character positioning result from the plurality of target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

2. The method of claim 1, wherein the license plate character detection model comprises an input layer, a plurality of dense blocks and transition layers, a convolutional layer and a target frame detection layer which are sequentially cascaded;

the size of the input image of the input layer is not more than 35 x 35;

the dense block and the transition layer are used for carrying out feature extraction and feature aggregation on the image;

the convolution kernel size of the convolution layer is 5 multiplied by 3;

the aspect ratio of the inspection frame of the target frame inspection layer includes 1, 2, 3, 5, 1/2, 1/3, and 1/5.

3. The method of claim 1, wherein the training of the license plate character detection model comprises:

inputting vehicle images for training into the license plate character detection model, and determining a plurality of target detection frames;

calculating loss values of the target detection frames according to a set target function, and updating parameters of the license plate character detection model by taking the loss values approaching a preset loss threshold as a target;

the vehicle images used for training comprise vehicle images with the license plates being regular in the forward direction, vehicle images with the license plates inclining to the right and vehicle images with the license plates inclining to the left.

4. The method of claim 3, wherein the objective function is:

wherein x is a preset matching indication matrix, c is a prediction confidence coefficient, L is a predicted position, g is a real position, a is a weight of position regression loss, and L_locIs L1 loss of loss function, L_confIs a softmax classification function.

5. The method of claim 1, wherein the step of coarsely locating the license plate of the vehicle image to be processed to obtain coarsely located license plate position information comprises:

processing the vehicle image to be processed through the trained Yolo model to obtain coarse license plate position information;

the Yolo model is obtained by training the vehicle image marked with the license plate frame as training data.

6. The method of claim 1, wherein the step of determining the result of locating the license plate characters from the plurality of target detection boxes comprises:

constructing a first list and a second list, wherein the first list is initialized to coordinate data comprising the plurality of target detection boxes, and the second list is initialized to be an empty list;

determining a first target detection box with the maximum confidence value from the first list;

removing the first target detection box from the first list and adding the first target detection box into a second list;

acquiring an intersection ratio of the target detection frame and a first target detection frame aiming at each target detection frame in the first list;

and removing the target detection frame with the intersection ratio value larger than a preset threshold value from the first list, returning to the step of determining the first target detection frame with the maximum confidence value from the first list, and determining the license plate character positioning result according to the target detection frame in the second list until the first list is empty.

7. The method of claim 1, wherein before processing the first feature image by the trained license plate character detection model, the method further comprises:

zooming the first characteristic image to a preset size to obtain a first characteristic image with the preset size;

and carrying out normalization processing on the colorimetric values and the brightness values of the first characteristic image with preset sizes.

8. A license plate character positioning device, comprising:

the license plate coarse positioning unit is used for performing license plate coarse positioning on a vehicle image to be processed to obtain coarse license plate position information;

the first characteristic image acquisition unit is used for intercepting a license plate character region of interest from the vehicle image based on the rough license plate position information to obtain a first characteristic image;

the target detection frame acquisition unit is used for processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

the positioning result acquisition unit is used for determining a license plate character positioning result from the target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

9. A license plate character locating apparatus, comprising: a memory and a processor;

the memory is used for storing programs;

the processor is configured to execute the program to implement the steps of the license plate character positioning method according to any one of claims 1 to 7.

10. A storage medium, comprising: when being executed by a processor, the computer program realizes the steps of the license plate character positioning method according to any one of claims 1-7.

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a license plate character positioning method and related devices.

Background

License Plate Recognition (VLPR) is an application of computer video image Recognition technology in Vehicle License Plate Recognition. License plate recognition is widely applied to highway vehicle management and parking lot management, and the license plate recognition technology becomes a main means for vehicle identity recognition.

In the management of vehicles on expressways, the license plate recognition technology is combined with an Electronic Toll Collection (ETC) to recognize vehicles, so that the vehicles passing by can realize automatic identification and automatic charging of vehicle identities without stopping when passing through a gate.

In the parking lot management, in order to improve the passing efficiency of vehicles at an entrance and an exit, the license plate recognition technology is combined with a charging system to carry out passing management on passing vehicles, and an unattended fast channel is built, so that passing vehicles can pass in and out a barrier gate without getting cards or stopping the vehicles.

Unlike character recognition in scanned documents, license plate characters in natural scenes have low contrast, different backgrounds, more bright interference, and the like. In addition, in order to allow the vehicles to pass through quickly and avoid the vehicles from blocking the entrance and exit, the speed of license plate recognition is important. For this reason, an efficient method is needed to identify the license plate of a moving car from a complex background.

The license plate character positioning is a key technology of license plate recognition and is responsible for extracting a license plate character area from a complex background. How to quickly and accurately position the license plate characters is very important, and the license plate recognition effect is directly influenced.

Disclosure of Invention

In view of this, the present application provides a license plate character positioning method and related devices, so as to accurately extract a license plate character region and achieve license plate character positioning.

In order to achieve the above object, a first aspect of the present application provides a license plate character positioning method, including:

carrying out coarse license plate positioning on a vehicle image to be processed to obtain coarse license plate position information;

based on the rough license plate position information, intercepting a license plate character region of interest from the vehicle image to obtain a first characteristic image;

processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

determining a license plate character positioning result from the plurality of target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

Preferably, the license plate character detection model comprises an input layer, a plurality of dense blocks, a transition layer, a convolution layer and a target frame detection layer which are sequentially cascaded;

the size of the input image of the input layer is not more than 35 x 35;

the dense block and the transition layer are used for carrying out feature extraction and feature aggregation on the image;

the convolution kernel size of the convolution layer is 5 multiplied by 3;

the aspect ratio of the inspection frame of the target frame inspection layer includes 1, 2, 3, 5, 1/2, 1/3, and 1/5.

Preferably, the process of training the license plate character detection model includes:

inputting vehicle images for training into the license plate character detection model, and determining a plurality of target detection frames;

calculating loss values of the target detection frames according to a set target function, and updating parameters of the license plate character detection model by taking the loss values approaching a preset loss threshold as a target;

the vehicle images used for training comprise vehicle images with the license plates being regular in the forward direction, vehicle images with the license plates inclining to the right and vehicle images with the license plates inclining to the left.

Preferably, the objective function is:

wherein x is a preset matching indication matrix, c is a prediction confidence coefficient, L is a predicted position, g is a real position, a is a weight of position regression loss, and L_locIs L1 loss of loss function, L_confIs a softmax classification function.

Preferably, the process of coarsely positioning the license plate of the vehicle image to be processed to obtain coarsely determined license plate position information includes:

processing the vehicle image to be processed through the trained Yolo model to obtain coarse license plate position information;

the Yolo model is obtained by training the vehicle image marked with the license plate frame as training data.

Preferably, the process of determining the license plate character positioning result from the plurality of target detection frames includes:

constructing a first list and a second list, wherein the first list is initialized to coordinate data comprising the plurality of target detection boxes, and the second list is initialized to be an empty list;

determining a first target detection box with the maximum confidence value from the first list;

removing the first target detection box from the first list and adding the first target detection box into a second list;

acquiring an intersection ratio of the target detection frame and a first target detection frame aiming at each target detection frame in the first list;

and removing the target detection frame with the intersection ratio value larger than a preset threshold value from the first list, returning to the step of determining the first target detection frame with the maximum confidence value from the first list, and determining the license plate character positioning result according to the target detection frame in the second list until the first list is empty.

Preferably, before the processing of the first feature image by the trained license plate character detection model, the method further includes:

zooming the first characteristic image to a preset size to obtain a first characteristic image with the preset size;

and carrying out normalization processing on the colorimetric values and the brightness values of the first characteristic image with preset sizes.

The present application provides in a second aspect a license plate character positioning device, including:

the license plate coarse positioning unit is used for performing license plate coarse positioning on a vehicle image to be processed to obtain coarse license plate position information;

the first characteristic image acquisition unit is used for intercepting a license plate character region of interest from the vehicle image based on the rough license plate position information to obtain a first characteristic image;

the target detection frame acquisition unit is used for processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

the positioning result acquisition unit is used for determining a license plate character positioning result from the target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

A third aspect of the present application provides a license plate character positioning apparatus, including: a memory and a processor;

the memory is used for storing programs;

the processor is used for executing the program to realize the steps of the license plate character positioning method.

A fourth aspect of the present application provides a storage medium comprising: when being executed by a processor, the computer program realizes the steps of the license plate character positioning method.

According to the technical scheme, the license plate is roughly positioned on the vehicle image to be processed to obtain the roughly-positioned license plate position information, and the license plate character region of interest can be obtained through roughly-positioned license plate position information. And then processing the images of the license plate character interesting regions through the trained license plate character detection model to obtain a plurality of candidate detection frames. Wherein, the candidate detection frames may overlap with each other. And finally, carrying out post-processing on the plurality of candidate detection frames, eliminating redundant candidate frames and determining a license plate character positioning result.

The license plate character detection model is specially configured, and the size of the convolution kernel of the convolution layer is configured to be matched with the shape characteristic of a license plate, so that the image characteristics of the license plate characters can be effectively extracted; the aspect ratio of the detection frame of the target frame detection layer is configured to be matched with the shape characteristic of the license plate, so that the license plate character text frame can be accurately captured.

Furthermore, the license plate character positioning method and the license plate character positioning system have the advantages that the license plate rough positioning module, the vehicle character detection model processing module and the model output post-processing module are cascaded, and license plate character positioning can be achieved accurately.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic diagram of a license plate character positioning method disclosed in an embodiment of the present application;

FIG. 2 is a schematic diagram of a network structure of a license plate character detection model disclosed in an embodiment of the present application;

FIG. 3 is a schematic diagram of a license plate character positioning device disclosed in an embodiment of the present application;

fig. 4 is a block diagram of a hardware structure of a license plate character positioning device disclosed in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, a license plate character positioning method provided in an embodiment of the present application may include the following steps:

and S100, acquiring position information of a roughly-determined license plate.

Specifically, the vehicle image to be processed is subjected to coarse license plate positioning to obtain coarse license plate positioning information.

The license plate coarse positioning is a method for carrying out license plate coarse positioning by combining an image space information method and a color space information method.

For example, a region containing a license plate is found from an image after the edge detection of the license plate, the region is extracted, the upper edge and the lower edge of the license plate are determined by utilizing the continuity characteristic of the horizontal integral projection of the license plate region, the feature that the number of white dots of the license plate occupies the dominance is utilized, a rectangle with a certain width is used for scanning from left to right, and finally the position with the largest number of white dots in the rectangle is the approximate position of the license plate region.

The position information of the roughly-determined license plate can be obtained through the rough positioning of the license plate, and the information can be applied to the precise positioning of the license plate, so that a good foundation is laid for the subsequent license plate positioning and identification.

And S200, acquiring a first characteristic image according to the position information of the roughly-determined license plate.

Specifically, based on the rough license plate position information in step S100, a license plate character region of interest is intercepted from the vehicle image to be processed, and a first feature image is obtained.

The rough license plate position information may be coordinate data of a license plate region. And intercepting an area possibly containing license plate characters from the vehicle image to be processed by roughly determining the license plate position information, and taking the area as a license plate character interested area to obtain a first characteristic image.

Through the processing of the step, the region to be detected is reduced from the whole vehicle image to be processed to the region of interest of the license plate characters, and the workload of subsequent processing is reduced.

And step S300, acquiring a plurality of target detection frames through a neural network model according to the first characteristic image.

Specifically, the first characteristic image is processed through the trained license plate character detection model, and a plurality of target detection frames are obtained.

Wherein the plurality of target detection frames may be detection frames of different sizes, and there may be different degrees of overlap between the target detection frames.

And step S400, determining a license plate character positioning result from a plurality of target detection frames.

Specifically, since the plurality of target detection frames output in step S300 may overlap to different degrees, redundant or relatively low-scoring target detection frames may be included. And determining the license plate character positioning result from the plurality of target detection frames, effectively removing redundant detection frames, and reserving the most target detection frame as the license plate character positioning result.

The license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

The license plate character region of interest can be obtained by roughly positioning the license plate of the vehicle image to be processed to obtain the roughly-determined license plate position information. And then processing the images of the license plate character interesting regions through the trained license plate character detection model to obtain a plurality of candidate detection frames. Wherein, the candidate detection frames may overlap with each other. And finally, carrying out post-processing on the plurality of candidate detection frames, eliminating redundant candidate frames and determining a license plate character positioning result.

The license plate character detection model is specially configured, and the size of the convolution kernel of the convolution layer is configured to be matched with the shape characteristic of a license plate, so that the image characteristics of the license plate characters can be effectively extracted; the aspect ratio of the detection frame of the target frame detection layer is configured to be matched with the shape characteristic of the license plate, so that the license plate character text frame can be accurately captured.

Furthermore, the license plate character positioning method and the license plate character positioning system have the advantages that the license plate rough positioning module, the vehicle character detection model processing module and the model output post-processing module are cascaded, and license plate character positioning can be achieved accurately.

In some embodiments of the present application, referring to fig. 2, the license plate character detection model mentioned in the above step S300 includes an input layer 11, a plurality of dense blocks 12 and a transition layer 13 (only one dense block and one transition layer are shown in the figure, for space limitation), a convolutional layer 14 and a target frame detection layer 15, which are sequentially cascaded. Wherein the size of the input image of the input layer 11 is not more than 35 × 35; the dense block 12 and the transition layer 13 are used for carrying out feature extraction and feature aggregation on the image; the convolution kernel size of convolution layer 14 is 5 × 3; the aspect ratio of the inspection frame of the target frame inspection layer 15 includes 1, 2, 3, 5, 1/2, 1/3, and 1/5.

The present inventors considered that the resolution of the input image does not need to be too large, and it is limited to 35 × 35, considering the number of downsampling times and the output size of the network as a whole. By limiting the size of the input image within 35 multiplied by 35, the total calculated amount of license plate character positioning can be compressed, and the inference efficiency of license plate detection is improved. In addition, experiments prove that the size can achieve the effect of accurate detection.

For two of the convolutional layers 14, a convolution kernel of 5 × 3 is used, specifically, kernel _ h is 3, kernel _ w is 5, pad _ h is 1, and pad _ w is 2. Compared with regular convolution kernels of 1 × 1, 3 × 3 and the like, the convolution kernel with irregular size better conforms to the shape of a long rectangle of license plate characters, and detection accuracy is improved.

For the detection frames in the target frame detection layer 15, a plurality of different aspect ratios are adopted so as to adapt to more license plate shapes, including the shapes of license plates of different specifications and the shapes of license plates of the same specification at different shooting angles.

In an alternative embodiment, the structure of each layer in the license plate character detection model mentioned in step S300 is specifically shown in table 1.

Table 1: network structure of license plate character detection model

The network structure of the license plate character detection model in table 1 is described in detail below.

Specifically, the first and second convolutional layers are 3 × 3 convolutional layers, and the first convolutional layer stride is 1, so that the detailed information can be kept as much as possible; the second convolution layer stride is 2, and downsampling is performed, so that the size of the feature can be reduced, and the subsequent calculation efficiency can be improved.

The network structure comprises 5 dense blocks and 4 corresponding transition layers. The dense blocks are used for feature extraction, and contain cross-layer connection, so that optimization efficiency is improved; the transition layer is the first conversion layer, which aims to converge the features of intensive block processing and to down-sample.

The third last convolution layer is a1 x 1 convolution layer and is used for converging the front information, exchanging channel information and changing the number of channels.

The last two convolutional layers abandon the convolutional layers of the original dense convolutional network (DenseNet), where kernel _ h is 1, kernel _ w is 1, and pad _ h is pad _ w is 1, and instead, the convolutional layers of kernel _ h is 3, kernel _ w is 5, pad _ h is 1, and pad _ w is 2 are used, so that the convolutional layers are more consistent with the shape of the long rectangle of the license plate character, and the detection accuracy is improved.

The target frame detection layer adopts a PriorBox layer of an ssd detection network, which is a layer absent in the original DenseNet and is additionally arranged for realizing the detection of the license plate character area in order to meet the requirement of license plate character positioning.

In some embodiments of the present application, the training process of the license plate character detection model in step S300 may include:

a1, inputting a vehicle image for training to a license plate character detection model, and determining a plurality of target detection frames;

a2, calculating loss values of the target detection frames according to a set target function, and updating parameters of the license plate character detection model by taking the loss values approaching a preset loss threshold as a target.

The vehicle images used for training comprise vehicle images with the license plates being regular in the forward direction, vehicle images with the license plates inclining to the right and vehicle images with the license plates inclining to the left.

Through the training process, the learnable parameters of the license plate character detection model are updated, so that the license plate character detection model can be suitable for license plate character positioning in various scenes.

In some embodiments of the present application, the objective function in a2 may be:

wherein c is the prediction confidence, l is the predicted position, g is the true position, and a is the weight of position regression loss; l is_locIs L1 loss of loss function, L_confIs a softmax classification function.

And x is a preset matching indication matrix. In particular toAnd for the ith default box and the jth group channel, if the two match, x_i,j1, otherwise, x_i,j0. The default box refers to a default set basic frame in the license plate character detection model, and the default box is used for specifying a basic value of frame change; the ground route refers to a labeled license plate character boundary box and is used for providing a label for training a license plate character detection model.

In some embodiments of the present application, the step S100 of performing coarse license plate positioning on the to-be-processed vehicle image to obtain coarse license plate position information may include:

processing the vehicle image to be processed through the trained Yolo model to obtain coarse license plate position information;

the Yolo model is obtained by training the vehicle image marked with the license plate frame as training data.

For example, firstly, a camera is used for collecting a picture containing a license plate, a license plate frame is labeled to obtain license plate coordinates, the license plate picture and a labeling file are used as a training data set, a Yolo model is trained, and the trained Yolo model is obtained.

And then deploying the trained Yolo model on an inference structure, inputting the image of the vehicle to be processed, and finally calculating the position information of the rough-determined license plate.

The step S400 may determine the license plate character positioning result from the target detection frames in various manners, such as a Non-Maximum Suppression (NMS) method and various deformation algorithms thereof. Based on this, in some embodiments of the present application, the process of determining the license plate character positioning result from the plurality of target detection frames in step S400 may include:

b1, constructing a first list and a second list, wherein the first list is initialized to include the coordinate data of the plurality of target detection boxes, and the second list is initialized to be an empty list;

b2, determining a first target detection box with the maximum confidence value from the first list;

b3, removing the first target detection box from the first list and adding the first target detection box into the second list;

b4, aiming at each target detection frame in the first list, acquiring the intersection ratio of the target detection frame and the first target detection frame;

b5, removing the target detection box with the intersection ratio larger than the preset threshold value from the first list, and returning to execute B2 until the first list is empty;

and B6, determining the license plate character positioning result according to the target detection frame in the second list.

The intersection ratio (IOU) is the intersection ratio of two frames, i.e. the intersection area of two frames is divided by the Union area, and the function is to define the coincidence degree of the two frames.

Through the processing procedure, redundant target detection frames can be removed, and the target detection frame with better detection structure evaluation is reserved.

For example, in an alternative embodiment, for the list B of the bounding box and its corresponding confidence S, the detection box M with the highest confidence is selected, removed from the set B and added to the list D. The intersection ratio of the rest detection boxes in the list B and M is larger than a threshold value N_tThe box of (a) is removed from list B and the process is repeated until list B is empty. Wherein, the bounding box refers to a rectangular box and is used for designating a prediction box; common threshold N_t0.3 to 0.5.

It will be appreciated that when the vehicle image to be processed itself contains a plurality of license plates, a plurality of detection boxes may remain in the list D. If a plurality of detection frames are reserved in the last list D, the detection frame with the highest confidence level S can be taken as a license plate character positioning result according to actual needs, and all the detection frames in the list D can also be taken as license plate character positioning results. And finally obtaining the best frame coordinate.

And processing the coordinates of the detection frame determined from the list D, and multiplying the coordinates by the width and the height of the input image respectively to obtain the coordinates relative to the license plate characters in the input image.

In some embodiments of the present application, before processing the first feature image through the trained license plate character detection model in step S200, the method may further include:

c1, zooming the first feature image to a preset size to obtain a first feature image with a preset size;

and C2, performing normalization processing on the colorimetric values and the brightness values of the first characteristic image with preset sizes.

The license plate character positioning device provided by the embodiment of the application is described below, and the license plate character positioning device described below and the license plate character positioning method described above can be referred to correspondingly.

Referring to fig. 3, a license plate character positioning device provided in an embodiment of the present application may include:

the license plate coarse positioning unit 21 is used for performing license plate coarse positioning on a vehicle image to be processed to obtain coarse license plate position information;

the first characteristic image acquisition unit 22 is configured to intercept a license plate character region of interest from the vehicle image based on the rough license plate position information to obtain a first characteristic image;

the target detection frame acquisition unit 23 is configured to process the first feature image through the trained license plate character detection model to obtain a plurality of target detection frames;

a positioning result obtaining unit 24, configured to determine a license plate character positioning result from the multiple target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

In some embodiments of the present application, the process of training the license plate character detection model in the target detection frame obtaining unit 23 may include:

inputting vehicle images for training into the license plate character detection model, and determining a plurality of target detection frames;

calculating loss values of the target detection frames according to a set target function, and updating parameters of the license plate character detection model by taking the loss values approaching a preset loss threshold as a target;

the vehicle images used for training comprise vehicle images with the license plates being regular in the forward direction, vehicle images with the license plates inclining to the right and vehicle images with the license plates inclining to the left.

In some embodiments of the present application, the process of the license plate coarse positioning unit 21 performing license plate coarse positioning on the vehicle image to be processed to obtain coarse license plate position information may include:

processing the vehicle image to be processed through the trained Yolo model to obtain coarse license plate position information;

the Yolo model is obtained by training the vehicle image marked with the license plate frame as training data.

In some embodiments of the present application, the process of determining the license plate character positioning result from the plurality of target detection frames by the positioning result obtaining unit 24 may include:

constructing a first list and a second list, wherein the first list is initialized to coordinate data comprising the plurality of target detection boxes, and the second list is initialized to be an empty list;

determining a first target detection box with the maximum confidence value from the first list;

removing the first target detection box from the first list and adding the first target detection box into a second list;

acquiring an intersection ratio of the target detection frame and a first target detection frame aiming at each target detection frame in the first list;

and removing the target detection frame with the intersection ratio value larger than a preset threshold value from the first list, returning to the step of determining the first target detection frame with the maximum confidence value from the first list, and determining the license plate character positioning result according to the target detection frame in the second list until the first list is empty.

In some embodiments of the present application, the license plate character locating device may further include an image preprocessing unit. The image preprocessing unit is configured to preprocess the first feature image before processing the first feature image through the trained license plate character detection model, where the preprocessing process may include:

zooming the first characteristic image to a preset size to obtain a first characteristic image with the preset size;

and carrying out normalization processing on the colorimetric values and the brightness values of the first characteristic image with preset sizes.

The license plate character positioning device provided by the embodiment of the application can be applied to license plate character positioning equipment. Optionally, fig. 4 shows a block diagram of a hardware structure for license plate character positioning, and referring to fig. 4, the hardware structure of the license plate character positioning device may include: at least one processor 31, at least one communication interface 32, at least one memory 33 and at least one communication bus 34.

In the embodiment of the present application, the number of the processor 31, the communication interface 32, the memory 33 and the communication bus 34 is at least one, and the processor 31, the communication interface 32 and the memory 33 complete the communication with each other through the communication bus 34;

the processor 31 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present application, etc.;

the memory 32 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (non-volatile memory) or the like, such as at least one disk memory;

wherein the memory 33 stores a program and the processor 31 may invoke the program stored in the memory 33, the program being for:

carrying out coarse license plate positioning on a vehicle image to be processed to obtain coarse license plate position information;

based on the rough license plate position information, intercepting a license plate character region of interest from the vehicle image to obtain a first characteristic image;

processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

determining a license plate character positioning result from the plurality of target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate. Alternatively, the detailed function and the extended function of the program may be as described above.

Embodiments of the present application further provide a storage medium, where a program suitable for execution by a processor may be stored, where the program is configured to:

carrying out coarse license plate positioning on a vehicle image to be processed to obtain coarse license plate position information;

based on the rough license plate position information, intercepting a license plate character region of interest from the vehicle image to obtain a first characteristic image;

processing the first characteristic image through the trained license plate character detection model to obtain a plurality of target detection frames;

determining a license plate character positioning result from the plurality of target detection frames;

the license plate character detection model is configured to have the capacity of extracting image features of an input image, performing convolution processing on the image features based on a convolution layer to obtain a second feature image, and determining a plurality of target detection frames from the second feature image through a target frame detection layer; and the size of the convolution kernel of the convolution layer and the aspect ratio of the detection frame of the target frame detection layer are matched with the shape characteristic of the license plate.

Alternatively, the detailed function and the extended function of the program may be as described above.

In summary, the following steps:

the license plate character region of interest can be obtained by roughly positioning the license plate of the vehicle image to be processed to obtain the roughly-determined license plate position information. And then processing the images of the license plate character interesting regions through the trained license plate character detection model to obtain a plurality of candidate detection frames. Wherein, the candidate detection frames may overlap with each other. And finally, carrying out post-processing on the plurality of candidate detection frames, eliminating redundant candidate frames and determining a license plate character positioning result.

The license plate character detection model is specially configured, and the size of the convolution kernel of the convolution layer is configured to be matched with the shape characteristic of a license plate, so that the image characteristics of the license plate characters can be effectively extracted; the aspect ratio of the detection frame of the target frame detection layer is configured to be matched with the shape characteristic of the license plate, so that the license plate character text frame can be accurately captured.

Furthermore, the license plate character positioning method and the license plate character positioning system have the advantages that the license plate rough positioning module, the vehicle character detection model processing module and the model output post-processing module are cascaded, and license plate character positioning can be achieved accurately.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, the embodiments may be combined as needed, and the same and similar parts may be referred to each other.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.