阅读说明：本技术 一种基于Faster-RCNN的船牌识别方法 (Ship plate identification method based on fast-RCNN ) 是由 宣琦 张鑫辉 翔云 于 2019-08-12 设计创作，主要内容包括：一种基于Faster-RCNN的船牌识别方法,包括以下步骤：1)数据获取：通过港口摄像头采集港口进出船舶的图片；2)数据处理：对原图裁剪获取船牌图片,对船牌图片进行裁剪获取单个文字图片；3)文字切割模型训练：用Faster-RCNN模型训练船牌切割；4)文字识别模型训练：用两个ResNet模型分别训练数字识别跟中文识别；5)模型测试：用训练好的Faster-RCNN模型对船牌图片进行切割,获得各个船牌文字图片的位置,面积,得分和顺序,通过面积跟得分筛选后,再基于船牌文字顺序传入对应ResNet模型,识别各个文字,得到完整船牌名。本发明检测速度快和准确率高,具有较高的实际应用价值。(A ship plate identification method based on fast-RCNN comprises the following steps: 1) data acquisition: acquiring pictures of ships entering and exiting a port through a port camera; 2) data processing: cutting an original image to obtain a ship plate picture, and cutting the ship plate picture to obtain a single character picture; 3) training a character cutting model: training the ship board to cut by using a fast-RCNN model; 4) training a character recognition model: respectively training digital recognition and Chinese recognition by using two ResNet models; 5) and (3) testing a model: cutting the ship plate pictures by using the trained fast-RCNN model to obtain the position, the area, the score and the sequence of each ship plate character picture, screening the ship plate character pictures by the area and the score, transmitting the ship plate character pictures into a corresponding ResNet model based on the ship plate character sequence, identifying each character, and obtaining a complete ship plate name. The invention has the advantages of high detection speed, high accuracy and higher practical application value.)

1. A ship plate identification method based on fast-RCNN is characterized by comprising the following steps:

s1: data acquisition: acquiring pictures of ships entering and exiting a port through a port camera;

s2: data processing: cutting an original image to obtain a ship plate picture, and cutting the ship plate picture to obtain a single character picture;

s3: training a character cutting model: training the ship board to cut by using a fast-RCNN model;

s4: training a character recognition model: respectively training digital recognition and Chinese recognition by using two ResNet models;

s5: and (3) testing a model: cutting the ship plate pictures by using the trained fast-RCNN model to obtain the size, score and sequence of each ship plate character picture, transmitting the ship plate character pictures into a corresponding ResNet model based on the ship plate character sequence, and identifying each character to obtain a complete ship plate name.

2. The fast-RCNN-based ship plate recognition method according to claim 1, wherein in the step S1, a port camera is used to capture a picture of a ship containing a ship plate, and the picture is stored in JPEG format; the pictures taken should cover a variety of known vessels.

3. The fast-RCNN-based ship board recognition method according to claim 1 or 2, wherein the step S2 includes the steps of:

s2-1: preparing a fast-RCNN data set: cutting JPEG pictures containing the ship plates to obtain ship plate pictures, taking partial pictures as a training set, manually marking character frames by using a software Labelimg manually, wherein the label types are 'number' and 'chicken', generating an xml file, making the pictures and the xml file into a VOC2007 data set format, and generating txt files of test, train and val;

s2-2: and cutting the character picture marked in the S2-1, converting the character picture into 48 × 48 size, converting the size into a gray-scale image and performing normalization processing.

4. The fast-RCNN-based ship board recognition method according to claim 1 or 2, wherein the step S3 includes the steps of:

s3-1: taking the ship plate pictures as input Conv layers of a neural network, extracting the characteristics of the input pictures through 13 Conv layers, 13 relu layers and 4 pooling layers, and outputting feature maps with the size of an original image 1/16;

s3-2: generating a plurality of anchor boxes through an RPN module, cutting the anchor boxes, judging the anchor boxes to belong to the foreground or the background through softmax, and correcting the anchors through frame regression to obtain more accurate proposals;

s3-3: the Rol Pooling layer obtains the propuls feature maps with fixed size by utilizing propulses generated by RPN and the feature maps obtained before;

s3-4: classification classifies the characteristic diagram of the suggestion frame, and specific Classification is carried out by utilizing a full connection layer and softmax; meanwhile, frame regression operation is completed by using L1 Loss to obtain the accurate position of the object, a Loss function is calculated, parameters of the whole network are updated at the same time, a training model is obtained, training Loss comprises classification Loss and regression Loss, and the calculation formula is as follows:

where k is an integer representing the subscript of each sample, p_kRepresenting the probability that the kth anchor is predicted to be the target,indicates the probability that the kth calibration box is predicted as the target, t_k＝{t_x，t_y，t_w，t_hDenotes a vector of four parameterized coordinates of the prediction box,is a vector of four parameterized coordinates of a calibration box, N_clsDenotes the normalized magnitude of the cls term, λ denotes the external weight, N_regIndicating that the reg terms are normalized to the number of anchor positions,represents a loss of classification defined as p_kRepresenting the probability of prediction to a certain class, p if the current sample is a positive sample_kIf the current sample is a negative sample, p is 1_k＝0，Is a label of the real data that has been labeled,represents the regression loss of the bounding box, and is defined as Smooth_L1(t-t^*)，Smooth_L1The definition of the function is

5. The fast-RCNN-based ship board recognition method according to claim 1 or 2, wherein in the step S4, two ResNet models are trained, respectively processing digital and chinese. The first ResNet model takes a digital name as a class mark, 10 classes are obtained by training and classifying, and the corresponding class is 0-9; the second ResNet model encodes Chinese into numbers as class labels, and the training classes are derived from the number of Chinese.

6. The method for ship identification based on fast-RCNN according to claim 1 or 2, wherein in step S4, the text image obtained in S2 is converted into 48 × 48 size, converted into gray scale image and normalized, and divided into two categories according to number and chinese, and chinese is first text-coded and then put into the folder of the corresponding category as the text classification data set, and two ResNet models are trained respectively.

7. The method as claimed in claim 1 or 2, wherein in step S5, the test set of pictures is inputted into the training model, the precise position, area and score of the characters in the ship picture are detected, the erroneous character pictures are deleted according to (0.01 original < area < 0.1 original) and (score > 0.8), and then the erroneous character pictures are transmitted to the corresponding ResNet model based on the ship character sequence to identify the specific characters, and finally the complete ship name is obtained.

Technical Field

The invention relates to the technical field of computer application and the field of target detection, in particular to a ship plate identification method based on fast-RCNN.

Background

In recent years, with the globalization of economy, the maritime import and export trade of China is rapidly developed, the number of ships entering and exiting a port is increased sharply, however, the monitoring and management level of the port cannot keep up with the rapid development of the economy of the port, the port and the ships cannot be effectively managed, the illegal berthing phenomenon of the ships is easy to occur, ship collision accidents are caused, casualties and property loss are caused, the marine environment is seriously polluted, the operation efficiency of the port is influenced, and the serious economic loss is caused. To ensure safe navigation, technologies such as communication between a ship and the shore and object recognition are becoming more and more important. The ship plate identification can avoid the risks to a great extent and help the port to realize effective management. Therefore, there is a need for an effective ship number plate recognition method for monitoring and recognizing ships entering and exiting a port, improving the port management level, and timely detecting and alarming when an illegal berthing phenomenon occurs.

However, ship text recognition in natural environments faces many difficulties and challenges: the ship plate character recognition is greatly challenged by the absence of uniform ship plate marks, non-uniform font specifications, complex weather and character occlusion and the like, so that the expected recognition effect cannot be achieved by only relying on digital image processing.

With the rapid development of computer technology, artificial intelligence technology is gradually changing our lives, and making our lives more convenient and efficient. And the recent rapid development of hardware technologies such as GPU and the like also makes the practical application of the deep neural network possible. Recently, researchers have proposed various methods for object detection using a deep neural network, in which fast-RCNN is being used in many object detection systems with its high accuracy and high recall rate, and the present invention also implements automatic identification of a ship's license plate based on the fast-RCNN in order to more accurately identify the ship's license plate.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides the ship plate identification method based on the Faster-RCNN, which can effectively improve the speed and the accuracy of ship plate identification.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a ship plate identification method based on fast-RCNN comprises the following steps:

s1: data acquisition: acquiring pictures of ships entering and exiting a port through a port camera;

s2: data processing: cutting an original image to obtain a ship plate picture, and cutting the ship plate picture to obtain a single character picture;

s3: training a character cutting model: training the ship board to cut by using a fast-RCNN model;

s4: training a character recognition model: respectively training digital recognition and Chinese recognition by using two ResNet models;

s5: and (3) testing a model: cutting the ship plate pictures by using a trained fast-RCNN model to obtain the position, area, score and sequence of each ship plate character picture, screening the area and the score, transmitting the ship plate character pictures into a corresponding ResNet model based on the ship plate character sequence, and identifying each character to obtain a complete ship plate name;

preferably, in step S1, the ship video shot by the port camera is taken every 0.5 second and stored in JPEG format.

The step S2 includes the steps of:

s2-1: preparing a fast-RCNN data set: cutting JPEG pictures containing the ship plates to obtain ship plate pictures, taking partial pictures as a training set, manually marking character frames by using a software Labelimg manually, wherein the label types are 'number' and 'chicken', generating an xml file, making the pictures and the xml file into a VOC2007 data set format, and generating txt files of test, train and val;

s2-2: and cutting the character picture marked in the S2-1, converting the character picture into 48 × 48 size, converting the size into a gray-scale image and performing normalization processing.

The step S3 includes the steps of:

s3-1: taking the ship plate pictures as input Conv layers of a neural network, extracting the characteristics of the input pictures through 13 Conv layers, 13 relu layers and 4 pooling layers, and outputting feature maps with the size of an original image 1/16;

s3-2: generating a plurality of anchor boxes through an RPN module, cutting the anchor boxes, judging the anchor boxes to belong to a foreground (characters) or a background through softmax, and correcting the anchors through frame regression to obtain more accurate proposals;

s3-3: the RoI Pooling layer obtains the propsal feature maps with fixed size by utilizing propsals generated by RPN and the feature mapping obtained before;

s3-4: classication classifies the generic feature maps, and the specific classes are classified by using the full connection layer and softmax; meanwhile, frame regression operation is completed by using L1 Loss to obtain the accurate position of the object, a Loss function is calculated, parameters of the whole network are updated at the same time, a training model is obtained, training Loss comprises classification Loss and regression Loss, and the calculation formula is as follows:

where k is an integer representing the subscript of each sample, p_kRepresenting the probability that the kth anchor is predicted to be the target,indicates the probability that the kth calibration box is predicted as the target, t_k＝{t_x，t_y，t_w，t_hDenotes a vector of four parameterized coordinates of the prediction box,is a vector of four parameterized coordinates of a calibration box, N_clsDenotes the normalized magnitude of the cls term, λ denotes the external weight, N_regIndicating that the reg terms are normalized to the number of anchor positions,represents a loss of classification defined as p_kRepresenting the probability of prediction to a certain class, p if the current sample is a positive sample_kIf the current sample is a negative sample, p is 1_k＝0，Is a label of the real data that has been labeled,represents the regression loss of the bounding box, and is defined as Smooth_L1(t-t^*)，Smooth_L1The definition of the function is

In step S4, two ResNet models are trained to process digital and chinese, respectively. The first ResNet model takes a digital name as a class mark, 10 classes are obtained by training and classifying, and the corresponding class is 0-9; the second ResNet model encodes Chinese into numbers as class labels, and the training classes are derived from the number of Chinese.

In step S4, the text image obtained in step S2 is converted into 48 × 48 size, converted into grayscale image, normalized, and divided into two categories according to numbers and chinese characters, and the chinese characters are first text-coded and then placed into folders with corresponding category labels as text classification data sets, and two ResNet models are respectively trained.

In step S5, the ship plate pictures are cut by the trained fast-RCNN model to obtain the area, score and sequence of each ship plate character picture, and after the area and score are screened, the ship plate character pictures are transmitted to the corresponding ResNet model to identify specific characters based on the ship plate character sequence, and finally the complete ship plate name is obtained.

The invention has the beneficial effects that: because the types, sizes and the like of the ship plates in natural scenes are different, all the ship plates are difficult to detect and identify accurately by using the traditional method. According to the invention, ship boards with different characteristics such as different types, different sizes and the like are selected as training data and labeled, so that abundant training samples are obtained. The invention utilizes the fast-RCNN model and the ResNet model to identify the ship plate, and compared with the traditional method for manually extracting the characteristics and the method utilizing the simple neural network, the invention saves the detection time and improves the accuracy.

Drawings

FIG. 1 is a flow chart of a fast-RCNN based ship plate identification method according to an embodiment of the present invention;

FIG. 2 is a flow chart of the fast-RCNN algorithm according to an embodiment of the present invention.

Fig. 3 is a flow chart of the ResNet algorithm according to an embodiment of the present invention.

FIG. 4 is a diagram illustrating an exemplary detection result according to an embodiment of the present invention.

Detailed Description

The following detailed description of embodiments of the invention is provided in connection with the accompanying drawings.

Referring to fig. 1 to 4, a ship plate recognition method based on fast-RCNN includes the following steps:

s1: data acquisition: acquiring pictures of ships entering and exiting a port through a port camera;

s2: data processing: cutting an original image to obtain a ship plate picture, and cutting the ship plate picture to obtain a single character picture;

s3: training a character cutting model: the plate cut was trained using the fast-RCNN model.

S4: training a character recognition model: respectively training digital recognition and Chinese recognition by using two ResNet models;

s5: and (3) testing a model: cutting the ship plate pictures by using a trained fast-RCNN model to obtain the position, area, score and sequence of each ship plate character picture, screening the area and the score, transmitting the ship plate character pictures into a corresponding ResNet model based on the ship plate character sequence, and identifying each character to obtain a complete ship plate name;

preferably, in step S1, the ship video shot by the port camera is taken every 0.5 second and stored in JPEG format.

The step S2 includes the steps of:

s2-1: preparing a fast-RCNN data set: cutting JPEG pictures containing the ship plates to obtain ship plate pictures, taking partial pictures as a training set, manually marking character frames by using a software Labelimg manually, wherein the label types are 'number' and 'chicken', generating an xml file, making the pictures and the xml file into a VOC2007 data set format, and generating txt files of test, train and val;

s2-2: and cutting the character picture marked in the S2-1, converting the character picture into 48 × 48 size, converting the size into a gray-scale image and performing normalization processing.

The step S3 includes the steps of:

s3-1: the ship plate pictures are used as input Conv layers of a neural network, the characteristics of the input pictures are extracted through 13 Conv layers, 13 relu layers and 4 posing layers, and feature maps with the size of the original pictures 1/16 are output.

S3-2: and generating a plurality of anchor boxes through an RPN module, cutting the anchor boxes, judging the anchor boxes to belong to the foreground (characters) or the background through softmax, and correcting the anchors through frame regression to obtain more accurate proposals.

S3-3: the RoI Pooling layer obtains the fixed-size propofol feature maps by using the proposals generated by the RPN and the feature maps obtained previously.

S3-4: classication classifies the generic feature maps, and the specific classes are classified by using the full connection layer and softmax; meanwhile, frame regression operation is completed by using L1 Loss to obtain the accurate position of the object, a Loss function is calculated, parameters of the whole network are updated at the same time, a training model is obtained, training Loss comprises classification Loss and regression Loss, and the calculation formula is as follows:

where k is an integer representing the subscript of each sample, p_kRepresenting the probability that the kth anchor is predicted to be the target,indicates the probability that the kth calibration box is predicted as the target, t_k＝{t_x，t_y，t_w，t_hDenotes a vector of four parameterized coordinates of the prediction box,is markVector of four parameterized coordinates of the fixed frame, N_clsDenotes the normalized magnitude of the cls term, λ denotes the external weight, N_regIndicating that the reg terms are normalized to the number of anchor positions,represents a loss of classification defined as p_kRepresenting the probability of prediction to a certain class, p if the current sample is a positive sample_kIf the current sample is a negative sample, p is 1_k＝0，Is a label of the real data that has been labeled,represents the regression loss of the bounding box, and is defined as Smooth_L1(t-t^*)，Smooth_L1The definition of the function is

In step S4, two ResNet models are trained to process digital and chinese, respectively. The first ResNet model takes a digital name as a class mark, 10 classes are obtained by training and classifying, and the corresponding class is 0-9; the second ResNet model encodes Chinese into numbers as class labels, and the training classes are derived from the number of Chinese.

In step S4, the text images processed in step S2 are divided into two categories (chinese is first coded by text) according to the number and the chinese, and then the two categories are placed into folders corresponding to the category labels as text classification data sets, and two ResNet models are respectively trained.

In step S5, the fast-RCNN model is used to cut the ship plate pictures to obtain the area, score and sequence of each ship plate character picture, and the wrong character picture is deleted according to the sum of (0.01 x original figure < area < 0.1 x original figure) (score > 0.8), and then transmitted to the corresponding ResNet model to identify the specific character based on the ship plate character sequence, and finally the complete ship plate name is obtained.

As described above, according to the embodiment of the invention, training data is made by marking the ship plate picture, the final model is obtained by training the fast-RCNN model and the ResNet model, and then the model is used for detecting the specific position of characters in the ship plate picture and classifying the characters to obtain the name of the ship plate. Compared with the traditional method for manually extracting the characteristics and the simple neural network method, the method has the advantages of high detection speed and high accuracy.

The above-mentioned embodiments are only preferred embodiments of the present invention, which are merely illustrative and not restrictive, and any person skilled in the art may substitute or change the technical solution of the present invention and the inventive concept thereof within the scope of the present invention.