阅读说明：本技术 一种基于局部特征的车辆检索方法 (Vehicle retrieval method based on local features ) 是由 蔡晓东 苏欣欣 于 2018-05-14 设计创作，主要内容包括：本发明公开了一种基于局部特征的车辆检索方法,具体公开了：用目标检测算法进行车脸和车窗的定位,得到车辆样本；样本通过一定的预处理,定位截取车脸和车窗,搭建多任务loss卷积神经网络的深度学习算法,训练得到CNN网络模型,实现车辆型号的分类和特征提取,用车脸及车窗显著特征进行匹配检索出目标车辆,检索出同一辆车,解决了现有技术中在无车牌的情况,由于同一车型外观无法分辨,难以实现准确检索车辆的技术缺陷。(the invention discloses a vehicle retrieval method based on local features, and particularly discloses: positioning the car face and the car window by using a target detection algorithm to obtain a car sample; the method comprises the steps of positioning and intercepting a car face and a car window through a certain pretreatment of a sample, building a deep learning algorithm of a multitask loss convolutional neural network, training to obtain a CNN network model, realizing classification and feature extraction of car models, matching and retrieving a target car by using the significant features of the car face and the car window, and retrieving the same car.)

1. a vehicle retrieval method based on local features is characterized by comprising the following steps:

Step S1: positioning of car faces and windows: detecting and positioning the car face and the car window without the license plate by a target detection algorithm Fast-RCNN positioning to obtain the coordinates of a local area;

Step S2: image preprocessing: carrying out detection, cutting and alignment pretreatment on the vehicle sample obtained in the step S1, converting the picture format into an LMDB format, and generating an average value reduction file;

step S3: establishing a CNN network model: training to obtain a CNN network model for feature extraction by constructing a multi-task loss double-branch network framework; combining the center loss and softmax on the loss layer of the network architecture, wherein the centers gathers samples of the same class by penalizing the offset of each class's sample and the sample center, which is denoted as:

wherein x is_iThe characteristic value of the ith picture is represented,Represents class y_iin the loss function, each iteration calculates the distance between the current data and the center, and then superimposes the distance on the center in the form of gradientUpdating of (1);

Softmax calculated the probability output for each class using the following formula,

Wherein x is_iis the i node value, y, of the softmax layer_iIs the ith output value, n is the node number of the softmax layer,

the last loss layer is a combination of the central loss function and softmax, the loss function being expressed as:

Step S4: extracting and comparing the characteristics of the car face and the car window: performing feature extraction on the sample to be retrieved and the vehicle retrieval library processed in the step S2 by using the CNN network model obtained by training in the step S3, then performing feature comparison, and obtaining the same vehicle by comparing and retrieving the features of the vehicle window and the vehicle face after identifying the images of the same vehicle type in a classified manner;

in the comparison stage of the car face and the car window, a similarity threshold value w needs to be set, the cosine distance is adopted to calculate the image similarity s, and the formula for calculating s is as follows:

The vehicle comparison stage is to compare the similarity of two vehicle images with a threshold value w, when s is larger than w, the similarity of the two images is high, the two images can be classified into the same vehicle type, and when s is smaller than or equal to w, the similarity of the two images input for comparison is low, and the two images belong to different vehicle types; if the vehicle type is the same, the vehicle window comparison is continued until the same vehicle with the highest similarity is searched out; and if the vehicle type is different, stopping the next vehicle window comparison.

2. The local feature-based vehicle retrieval method of claim 1, wherein the CNN network model in step S3 includes a car face classification model and a car window target extraction model.

3. The local feature-based vehicle retrieval method of claim 1, wherein the network architecture in the step S3 is a deep network architecture or a shallow network architecture.

4. the local feature-based vehicle retrieval method according to claim 1, wherein in step S4, it is determined that the comparison accuracy is highest when the similarity threshold of the car face comparison is determined to be 0.52 and the similarity threshold of the car window comparison is determined to be 0.71.

Technical Field

the invention relates to the field of image processing and retrieval, in particular to a vehicle retrieval method based on local features.

Background

with the continuous expansion and updating of the database of the traffic monitoring system, target vehicles can be accurately searched in the large-scale database, and the method is very helpful for social applications such as security departments, public security investigation criminals and the like. Many vehicles suspected of illegal crimes often have no license plate (including fake license plate, forged license plate, shielded license plate and the like), and the license plate identification-based method cannot effectively identify and search the vehicles. At present, vehicle identification and retrieval research is mainly focused on identifying a few types of vehicle models such as large trucks, medium automobiles and small cars. As image recognition research advances, research on the recognition of specific vehicle models, such as audi Q5, speed S500, etc., gradually appears, and since the appearance of the same vehicle model is indistinguishable, it is difficult to retrieve the same vehicle without a license plate.

Because the car face can effectively express the appearance attribute of the car, the car model can be accurately identified through classification based on the car face characteristics, the car window has bright local characteristics due to the preference of a car owner, and the special marks (hanging decorations, a decoration, a tissue box and an annual inspection mark) enable the cars with the same model to be distinguished.

Therefore, it is an urgent technical problem to be solved by those skilled in the art to provide a method for identifying and searching a unlicensed vehicle through a vehicle face and a vehicle window image.

disclosure of Invention

in view of the above, the invention provides a vehicle retrieval method based on local features, which includes positioning of a vehicle face and a vehicle window, image preprocessing, establishment of a CNN network model, extraction and comparison of the features of the vehicle face and the vehicle window, and realizes efficient and accurate retrieval in a large-scale vehicle database.

In order to achieve the purpose, the invention adopts the following technical scheme:

a vehicle retrieval method based on local features comprises the following steps:

Step S1: positioning of car faces and windows: positioning by a target detection algorithm Fast-RCNN, detecting a vehicle from a traffic checkpoint monitoring video by utilizing a moving target detection technology, detecting and positioning a vehicle face and a vehicle window without a license plate, positioning and intercepting the vehicle face and the vehicle window, and obtaining the coordinates of a local area;

Step S2: image preprocessing: carrying out detection, cutting and alignment pretreatment on the vehicle sample obtained in the step S1, converting the picture format into an LMDB format, and generating an average value reduction file;

step S3: establishing a CNN network model: training to obtain a CNN network model for feature extraction by constructing a multi-task loss double-branch network framework; the center loss and softmax are combined on the loss layer of the network architecture, where the centerlos forces samples of the same class to converge as much as possible by penalizing the samples of each class and the offset of the sample center. It is expressed as:

Wherein x is_iThe characteristic value of the ith picture is represented,represents class y_iIn the loss function, each iteration calculates the distance between the current data and the center, and then superimposes the distance on the center in the form of gradientUpdating of (1);

softmax calculated the probability output for each class using the following formula,

Wherein x is_iIs the i node value, y, of the softmax layer_iis the ith output value, n is the node number of the softmax layer,

The last loss layer is a combination of the central loss function and softmax, the loss function being expressed as:

Step S4: extracting and comparing the characteristics of the car face and the car window: performing feature extraction on the sample to be retrieved and the vehicle retrieval library processed in the step S2 by using the CNN network model obtained by training in the step S3, then performing feature comparison, and obtaining the same vehicle by comparing and retrieving the features of the vehicle window and the vehicle face after identifying the images of the same vehicle type in a classified manner;

In the comparison stage of the car face and the car window, a similarity threshold value w needs to be set, the cosine distance is adopted to calculate the image similarity s, and the formula for calculating s is as follows:

The vehicle comparison stage is to compare the similarity of two vehicle images with a threshold value w, when s is larger than w, the similarity of the two images is high, the two images can be classified into the same vehicle type, and when s is smaller than or equal to w, the similarity of the two images input for comparison is low, and the two images belong to different vehicle types; if the vehicle type is the same, the vehicle window comparison is continued until the same vehicle with the highest similarity is searched out; and if the vehicle type is different, stopping the next vehicle window comparison.

preferably, the CNN network model in step S3 includes a car face classification model and a car window target extraction model.

preferably, the network architecture in the step S3 is a deep network architecture or a shallow network architecture.

Preferably, in step S4, it is determined that the comparison accuracy is highest when the similarity threshold of the car face comparison is determined to be 0.52 and the similarity threshold of the car window comparison is determined to be 0.71.

the method has the beneficial effects that the method adopts a main algorithm flow of the convolutional neural network, and provides a vehicle retrieval method based on local characteristics. Positioning the face and the vehicle window by using a target detection algorithm to obtain a vehicle sample; the method comprises the steps of positioning and intercepting a car face and a car window through a certain pretreatment of a sample, building a deep learning algorithm of a multitask loss convolutional neural network, training to obtain a CNN network model, realizing classification and feature extraction of car models, matching and retrieving a target car by using the significant features of the car face and the car window, and retrieving the same car.

drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a schematic diagram of a retrieval flow framework of the present invention;

FIG. 2 is a multi-tasking loss dual-leg network architecture;

FIG. 3 is a flow chart of the Fast-RCNN algorithm.

Detailed Description

the technical solutions in the embodiments of the present invention will be fully and clearly described below with reference to the accompanying drawings in the embodiments of the present invention.

the vehicle retrieval method based on the local features is a retrieval flow frame schematic diagram based on the local features as shown in fig. 1, and comprises the steps of firstly positioning a vehicle face and a vehicle window, preprocessing an image, and establishing a CNN network model to obtain a vehicle face classification model and a vehicle window target extraction network model. And then, extracting characteristic information by using a CNN network model for comparison, screening out vehicles of the same vehicle type, and finally retrieving the same vehicle. The specific implementation process is as follows:

step S1: and the vehicle image is positioned by utilizing a target detection Fast-RCNN algorithm to obtain a vehicle face and vehicle window sample. The target detection algorithm based on deep learning is more efficient than the traditional license plate recognition detection and recognition, and can detect and position the car face without the license plate and obtain the vehicle face by intercepting the obtained coordinates of the local area. FIG. 3 is a flow chart of the Fast-RCNN algorithm.

step S2: and preprocessing the vehicle sample such as detection, cutting, alignment and the like, and converting the picture format into the LMDB format. And averaging all the training samples to generate a mean value file mean.

Step S3: and constructing a multi-task loss-based double-branch network framework, and training to obtain a CNN network model for feature extraction. The deep learning network framework based on CNNs can build a deep network framework and can also build a shallow network framework, the invention builds a shallow double-branch multitask loss network framework, as shown in a network framework schematic diagram shown in FIG. 2, wherein a single branch comprises three convolution-pooling layers, two full-connection layers and a loss layer, the number of convolution layers is 32/32/64, the size of the convolution layer is 5 x 5, and the step length is 1; the core sizes of the three pooling layers are all 3 x 3, the step length is 2, and the three pooling layers are all maximum pooling layers; the first full-connection layer sets the number of nodes to be 4096, the second full-connection layer sets the number of nodes to be 1024, the weight initialization parameters of the two full-connection layers are xavier, the bias initialization parameter is constant, and one loss layer is softmax; the merging branch part comprises two full-connection layers and a loss layer, the loss layer adopts the combination of a central loss function and softmax, and the loss value formula is calculated as follows:

step S4: as shown in a vehicle retrieval flow chart based on local features in fig. 1, a CNN network model trained in step S3 is used to perform feature extraction on the sample to be retrieved and the vehicle retrieval library processed in step S2, and then feature comparison is performed, after images of the same vehicle type are classified and identified, the same vehicle is retrieved by comparing vehicle window and vehicle face features;

in the comparison stage of the car face and the car window, a similarity threshold value w needs to be set, the cosine distance is adopted to calculate the image similarity s, and the formula for calculating s is as follows:

The vehicle comparison stage is to compare the similarity of two vehicle images with a threshold value w, when s is larger than w, the similarity of the two images is high, the two images can be classified into the same vehicle type, and when s is smaller than or equal to w, the similarity of the two images input for comparison is low, and the two images belong to different vehicle types; if the vehicle type is the same, the vehicle window comparison is continued until the same vehicle with the highest similarity is searched out; if the vehicle type is different, the next vehicle window comparison can be stopped, wherein the highest accuracy is determined when the similarity threshold value of the vehicle face comparison is determined to be 0.52 and the similarity threshold value of the vehicle window comparison is determined to be 0.71.

the previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.