阅读说明：本技术 车辆识别码的检测方法、装置及计算机设备 (Vehicle identification code detection method and device and computer equipment ) 是由 周康明 谷维鑫 于 2019-09-18 设计创作，主要内容包括：本申请涉及一种车辆识别码的检测方法、装置及计算机设备,该方法包括包括：获取车辆识别码的待检测图像和车辆识别码的参考字符串；通过目标检测模型检测待检测图像中是否存在车辆识别码区域图像,若存在,从待检测图像中提取得到车辆识别码区域图像；通过字符实例分割模型对车辆识别码区域图像进行识别分割,并根据车辆识别码区域图像的识别分割结果生成车辆识别码区域图像所对应的字符串；将车辆识别码区域图像所对应的字符串与车辆识别码的参考字符串进行比对；根据比对的结果,记录车辆识别码的第一检测结果,解决了传统技术中存在无法准确地对车辆识别码进行检测的技术问题,提升了车辆识别码检测的准确率。(The application relates to a method, a device and computer equipment for detecting a vehicle identification code, wherein the method comprises the following steps: acquiring an image to be detected of a vehicle identification code and a reference character string of the vehicle identification code; detecting whether a vehicle identification code regional image exists in an image to be detected or not through a target detection model, and if so, extracting the vehicle identification code regional image from the image to be detected; identifying and segmenting the vehicle identification code area image through a character instance segmentation model, and generating a character string corresponding to the vehicle identification code area image according to an identification and segmentation result of the vehicle identification code area image; comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code; according to the comparison result, the first detection result of the vehicle identification code is recorded, the technical problem that the vehicle identification code cannot be accurately detected in the traditional technology is solved, and the accuracy of vehicle identification code detection is improved.)

1. A method for detecting a vehicle identification code, the method comprising:

acquiring an image to be detected of a vehicle identification code and a reference character string of the vehicle identification code;

detecting whether a vehicle identification code regional image exists in the image to be detected or not through a target detection model, and if so, extracting the vehicle identification code regional image from the image to be detected;

identifying and segmenting the vehicle identification code area image through a character instance segmentation model, and generating a character string corresponding to the vehicle identification code area image according to an identification and segmentation result of the vehicle identification code area image;

comparing a character string corresponding to the vehicle identification code area image with a reference character string of the vehicle identification code;

and recording a first detection result of the vehicle identification code according to the comparison result.

2. The method according to claim 1, wherein after the acquiring the image to be detected of the vehicle identification code and the reference character string of the vehicle identification code, the method further comprises:

acquiring the character length of a reference character string of the vehicle identification code;

judging whether the character length of the reference character string meets a preset condition or not;

and recording a second detection result of the vehicle identification code according to a judgment result of whether the character length meets the preset condition.

3. The method according to claim 2, wherein the detecting whether the image to be detected includes a vehicle identification code region image through a target detection model, and if so, extracting the vehicle identification code region image from the image to be detected includes:

detecting the position data of the vehicle identification code area image in the image to be detected through a target detection model, and judging whether the vehicle identification code area image exists in the image to be detected or not;

recording a third detection result of the vehicle identification code according to a judgment result of whether the vehicle identification code area image exists;

and if the vehicle identification code regional image exists in the image to be detected, extracting the image to be detected according to the position data to obtain the vehicle identification code regional image.

4. The method of claim 3, wherein the recognition result of the vehicle identification code comprises the position of each character in the vehicle identification code area image, and the vehicle identification code is stored with a corresponding rubbing film picture in advance; after the generating of the character string corresponding to the vehicle identification code area image according to the identification and segmentation result of the vehicle identification code, the method further comprises:

dividing the vehicle identification code area image into each character in the vehicle identification code according to the position of each character in the vehicle identification code area image;

comparing each character obtained by segmentation with each character in the rubbing film picture, and judging whether each character obtained by segmentation is tampered;

and recording a fourth detection result of the vehicle identification code according to a judgment result of whether each character obtained by segmentation is tampered.

5. The method of claim 4, wherein the generating of each character in the rubbing film picture comprises:

acquiring a rubbing film picture corresponding to the vehicle identification code;

identifying the rubbing film picture through a character instance segmentation model to obtain each character in the rubbing film picture and the position of each character in the rubbing film picture;

and dividing the rubbing film picture into all characters in the rubbing film picture according to the positions of all characters in the rubbing film picture.

6. The method of claim 4, further comprising:

performing statistical analysis on the first detection result, the second detection result, the third detection result and the fourth detection result;

when the first detection result, the second detection result, the third detection result and the fourth detection result are all passed detection, generating a detection result that the vehicle identification code passes the detection;

and when one or more of the first detection result, the second detection result, the third detection result and the fourth detection result is/are not passed, generating a detection result that the vehicle identification code is not passed.

7. The method according to any one of claims 1 to 6, wherein the identifying and segmenting the vehicle identification code region image through a character instance segmentation model and generating a character string corresponding to the vehicle identification code region image according to the identifying and segmenting result of the vehicle identification code region image comprises:

carrying out target detection on the vehicle identification code area image through a character example segmentation model, and determining each character and the position of each character in the vehicle identification code area image;

carrying out target classification on the vehicle identification code area image through a character example segmentation model, and determining the category of each character in the vehicle identification code area image;

performing pixel-level target segmentation on the vehicle identification code region image through a character instance segmentation model to obtain a plurality of pixel-level segmentation objects;

selecting each character corresponding to the vehicle identification code from the plurality of pixel level segmentation objects according to each character in the vehicle identification code area image and the category of each character;

and sequencing the characters according to the positions and the types of the characters in the vehicle identification code area image to generate a character string corresponding to the vehicle identification code area image.

8. The method according to any one of claims 1 to 6, wherein the generating of the character instance segmentation model comprises:

acquiring a sample set of the vehicle identification code area image; the sample set of the vehicle identification code area images comprises a plurality of vehicle identification code area images, and the positions and the types of characters in the vehicle identification code area images are labeled;

and training the character example segmentation model according to the vehicle identification code area images, the positions of the characters in the vehicle identification code area images and the classes corresponding to the characters in the vehicle identification code area images.

9. The method of claim 8, wherein the generating of the sample set of vehicle identification code region images comprises:

obtaining a sample set of original vehicle identification code images, the sample set of original vehicle identification code images comprising a plurality of the original vehicle identification code images;

acquiring position data of a vehicle identification code area in each original vehicle identification code image;

extracting each vehicle identification code area image from each original vehicle identification code image according to the position data of the vehicle identification code area;

marking the position and the category of each character in each vehicle identification code area image by adopting a character edge tracing method;

carrying out data expansion operation on each marked vehicle identification code area image;

and generating a sample set of the vehicle identification code area images according to the marked vehicle identification code area images and the vehicle identification code area images obtained through data expansion operation.

10. The method of claim 9, wherein the data augmentation operations comprise at least one of layout transformation, bending transformation, rotation, and translation.

11. An apparatus for detecting a vehicle identification code, the apparatus comprising:

the device comprises an acquisition module, a detection module and a processing module, wherein the acquisition module is used for acquiring an image to be detected of a vehicle identification code and a reference character string of the vehicle identification code, and the image to be detected comprises a vehicle identification code area image;

the detection module is used for detecting whether a vehicle identification code regional image exists in the image to be detected through a target detection model, and if so, extracting the vehicle identification code regional image from the image to be detected;

the recognition module is used for recognizing and segmenting the vehicle identification code area image through a character instance segmentation model and generating a character string corresponding to the vehicle identification code area image according to the recognition and segmentation result of the vehicle identification code area image;

the comparison module is used for comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code;

and the recording module is used for recording a first detection result of the vehicle identification code according to the comparison result.

12. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 10 are implemented by the processor when executing the computer program.

Technical Field

The present disclosure relates to the field of vehicle detection, and in particular, to a method and an apparatus for detecting a vehicle identification code, and a computer device.

Background

With the continuous development of social economy and the continuous improvement of the living standard of people, the quantity of motor vehicles in cities is rapidly increased. The workload of annual inspection of motor vehicles is also rapidly increased. The detection and identification of the vehicle identification code are one of important items for vehicle annual inspection.

The Vehicle Identification Number (VIN) is a multi-character code serving as a unique Identification code of the Vehicle, and includes information such as a manufacturer, a year, a Vehicle type, a Vehicle body type and code, an engine code, an assembly location, and the like of the Vehicle. Typically, the vehicle identification code consists of a 17-digit character, comprising letters and numbers, colloquially known as a seventeen-digit code.

However, the conventional technology has a technical problem that the vehicle identification code cannot be accurately detected.

Disclosure of Invention

In view of the above, it is necessary to provide a method and an apparatus for detecting a vehicle identification code, and a computer device, for solving the technical problem that the vehicle identification code cannot be accurately detected in the conventional technology.

A method of detecting a vehicle identification code, the method comprising: acquiring an image to be detected of a vehicle identification code and a reference character string of the vehicle identification code; detecting whether a vehicle identification code regional image exists in the image to be detected or not through a target detection model, and if so, extracting the vehicle identification code regional image from the image to be detected; identifying and segmenting the vehicle identification code area image through a character instance segmentation model, and generating a character string corresponding to the vehicle identification code area image according to an identification and segmentation result of the vehicle identification code area image; comparing a character string corresponding to the vehicle identification code area image with a reference character string of the vehicle identification code; and recording a first detection result of the vehicle identification code according to the comparison result.

According to the detection method of the vehicle identification code, the vehicle identification code area images are respectively identified through the character instance segmentation model, and the character strings corresponding to the vehicle identification code area images are generated according to the identification segmentation result of the vehicle identification code area images, so that the adjacent same characters in the image to be detected of the vehicle identification code are effectively identified, the technical problem that the vehicle identification code cannot be accurately detected in the traditional technology is solved, and the accuracy of vehicle identification code detection is improved.

Drawings

FIG. 1 is a diagram of an exemplary embodiment of a vehicle identification code detection method;

FIG. 2a is a schematic flow chart illustrating a method for detecting a vehicle identification code according to one embodiment;

FIG. 2b is a diagram illustrating a vehicle identification code region in an image to be detected in one embodiment;

FIG. 3 is a schematic flow chart illustrating a method for detecting a vehicle identification code according to one embodiment;

FIG. 4 is a schematic flow chart illustrating a method for detecting a vehicle identification code according to one embodiment;

FIG. 5 is a schematic flow chart illustrating a method for detecting a vehicle identification code according to one embodiment;

FIG. 6 is a schematic flow chart illustrating the generation of characters in a film print in one embodiment;

FIG. 7a is a schematic flow chart illustrating a method for detecting a vehicle identification code according to one embodiment;

FIG. 7b is a diagram of a network structure of a character instance segmentation model in one embodiment;

FIG. 8 is a schematic flow chart diagram illustrating the generation of a sample set of vehicle identification code region images in one embodiment;

FIG. 9 is a block diagram showing a configuration of a vehicle identification code detection device according to an embodiment;

FIG. 10 is a diagram showing an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Referring to fig. 1, a schematic diagram of an application environment according to an embodiment of the present application is provided. The application environment may include: a first computer device 110, a second computer device 120, and an image acquisition device 130. The first Computer device 110 and the second Computer device 120 refer to electronic devices with strong data storage and computation capabilities, for example, the first Computer device 110 and the second Computer device 120 may be a PC (Personal Computer) or a server. Specifically, the image acquisition device 130 acquires an image of the vehicle identification code of the vehicle to obtain an image to be detected of the vehicle identification code, and sends the image to be detected of the vehicle identification code to the first computer device 110 through network connection. Before detecting the image to be detected, a technician is required to construct the target detection model on the second computer device 120, and train the constructed target detection model through the second computer device 120. The technician may also build a character instance segmentation model on the second computer device 120 and train the built character instance segmentation model through the second computer device 120. The trained target detection model and the trained character instance segmentation model can be issued from the second computer device 120 to the first computer device 110, the first computer device 110 can detect an image to be detected of the vehicle identification code provided by the user by adopting the target detection model, judge whether the image to be detected has a vehicle identification code region image, and if so, extract the vehicle identification code region image from the image to be detected; the vehicle identification code area image can be identified and segmented by adopting the character instance segmentation model to obtain the identification segmentation result of the vehicle identification code, and the character string corresponding to the vehicle identification code area image is generated according to the identification segmentation result of the vehicle identification code; then, comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code; and then recording a first detection result of the vehicle identification code according to the comparison result. It is understood that the first computer device 110 may also take the form of a terminal, which may be an electronic device such as a cell phone, a tablet, an e-book reader, a multimedia player device, a wearable device, a PC, etc. And the terminal finishes the detection work of the image to be detected of the vehicle identification code through the target detection model and the character instance segmentation model. Of course, the first computer device 110 may also be integrated with the image acquisition device 130.

In one embodiment, as shown in fig. 2a, a method for detecting a vehicle identification code is provided, which is described by taking the method as an example applied to the first computer device 110 in fig. 1, and includes the following steps:

s210, acquiring an image to be detected of the vehicle identification code and a reference character string of the vehicle identification code.

The Vehicle Identification Number (VIN) is a multi-character code serving as a unique Identification code of the Vehicle, and includes information such as a manufacturer, a year, a Vehicle type, a Vehicle body type and code, an engine code, an assembly location, and the like of the Vehicle. The image to be detected is an image which is obtained by shooting a vehicle identification code area through image acquisition equipment and needs to be subjected to vehicle identification code detection, and the image to be detected can only comprise the vehicle identification code area image, can not comprise the vehicle identification code area image and only comprise some background images, and can also comprise the vehicle identification code area image and some background images. Since the vehicle identification code is composed of several bit characters, a reference character string of the vehicle identification code, which is a criterion for judging whether the vehicle identification code in the image to be detected passes the detection, is stored in advance. It will be appreciated that the reference string of vehicle identification codes may be pre-stored locally on the first computer device or on a server communicatively connected to the first computer device.

Specifically, the image acquisition device is used for acquiring the image of the vehicle identification code of the vehicle, and sending the image to be detected of the vehicle identification code to the first computer device in a wired connection mode or a wireless connection mode, and the first computer device is used for acquiring the image to be detected of the vehicle identification code. The image to be detected of the vehicle identification code can also be stored in advance in the first computer device locally or in a server in communication connection with the first computer device, and the first computer device obtains the image to be detected from the server locally or in communication connection with the first computer device. Likewise, the reference string of the vehicle identification code may be obtained from a server local to the first computer device or communicatively connected to the first computer device.

S220, detecting whether a vehicle identification code region image exists in the image to be detected through the target detection model, and if so, extracting the vehicle identification code region image from the image to be detected.

The object detection model refers to a machine learning model for segmenting an object of interest (such as a vehicle identification code) from an image to be detected. For example, the object detection model may be a deep learning based SSD (single shot multi box detection) object detection algorithm model, and the SSD may be a single deep neural network that detects the vehicle id region in the image to be detected through a rectangular frame, as shown in fig. 2 b. The vehicle identification code region image is a partial image corresponding to the vehicle identification code in the image to be detected. Specifically, after an image to be detected is input into a target detection model, whether a vehicle identification code area image exists in the image to be detected is detected, and if the vehicle identification code area image exists, the image to be detected is extracted or intercepted to obtain the vehicle identification code area image.

And S230, identifying and segmenting the vehicle identification code area image through the character instance segmentation model, and generating a character string corresponding to the vehicle identification code area image according to the identification and segmentation result of the vehicle identification code area image.

The character instance segmentation model is a machine learning model for recognizing the outline of each character in the vehicle identification code at a pixel level. Specifically, a vehicle identification code area image detected by the target detection model is input into a character instance segmentation model, the vehicle identification code area image is identified by the character instance segmentation model, and the characters included in the vehicle identification code area image are identified. When the vehicle identification code area image comprising continuous same characters is identified by using the character example segmentation model, the vehicle identification code area image is subjected to target detection at the same time, so that the position of each character in the vehicle identification code is known, and the continuous same characters in the vehicle identification code are distinguished one by one. Then, for the vehicle identification code comprising continuous same characters, the adjacent same characters in the vehicle identification code area image can be identified through the character instance segmentation model. After the characters of the vehicle identification code in the vehicle identification code area image are identified, the characters of the vehicle identification code are divided according to the identified characters of the vehicle identification code to generate the character string corresponding to the vehicle identification code area image.

S240, comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code.

And S250, recording a first detection result of the vehicle identification code according to the comparison result.

Wherein the first detection result is used for indicating whether the vehicle identification code identified from the vehicle identification code area image passes the detection. Specifically, each character of the vehicle identification code in the vehicle identification area image is identified by using the character instance segmentation model, and a character string corresponding to the vehicle identification code area image is generated. In order to determine whether the image to be detected passes the audit, it is necessary to determine whether the character string corresponding to the vehicle identification code region image is consistent with the reference character string of the vehicle identification code. And comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code, recording a first detection result of the vehicle identification code as passing detection if the character string corresponding to the vehicle identification code area image is consistent with the reference character string of the vehicle identification code, and recording the first detection result of the vehicle identification code as not passing detection if the character string corresponding to the vehicle identification code area image is inconsistent with the reference character string of the vehicle identification code.

For example, as shown in fig. 2b, the vehicle VIN code generally consists of 17-bit characters, and the characters forming the vehicle VIN code include 34 characters "0-9", "a-N", "P", and "R-Z". When these characters are arranged and combined to form the vehicle VIN code, a plurality of identical characters often appear in succession. For example, a vehicle VIN code is: LA9BAGGV0FHXCF 049. For the vehicle VIN code containing continuous same characters (such as GG), the conventional technology is used for identification, and the identification result is as follows: LA9BAGV0FHXCF 049. The identification result of the traditional technology is compared with the vehicle VIN code, so that the traditional technology cannot identify the vehicle VIN code containing continuous same characters, and the vehicle identification code cannot be accurately detected. In the embodiment, the adjacent same characters in the vehicle identification code region image are identified through the character instance segmentation model, and the identification result is as follows: LA9BAGGV0FHXCF049, comparing the recognition result of the present embodiment with the vehicle VIN code, it can be seen that the method in the present embodiment can recognize the vehicle VIN code containing consecutive identical characters, thereby accurately detecting the vehicle identification code.

In the embodiment, the image to be detected of the vehicle identification code and the reference character string of the vehicle identification code are obtained; detecting whether a vehicle identification code regional image exists in an image to be detected or not through a target detection model, and if so, extracting the vehicle identification code regional image from the image to be detected; identifying and segmenting the vehicle identification code area image through a character instance segmentation model, and generating a character string corresponding to the vehicle identification code area image according to an identification and segmentation result of the vehicle identification code area image; comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code; and recording a first detection result of the vehicle identification code according to the comparison result. The method and the device realize effective recognition of the same adjacent characters in the image to be detected of the vehicle identification code, thereby solving the technical problem that the vehicle identification code cannot be accurately detected in the traditional technology, and improving the accuracy of vehicle identification code detection.

In one embodiment, as shown in fig. 3, after acquiring the image to be detected of the vehicle identification code and the reference character string of the vehicle identification code, the method further comprises the steps of:

s310, acquiring the character length of a reference character string of the vehicle identification code;

s320, judging whether the character length of the reference character string meets a preset condition or not;

and S330, recording a second detection result of the vehicle identification code according to a judgment result of whether the character length meets the preset condition.

Wherein the reference character string has a certain number of character components, i.e. has a certain character length. In order to ensure the consistency of the detection standard and improve the reliability of the detection result, after the reference character string of the vehicle identification code is acquired, the reference character string needs to be checked, for example, the length of the reference character string needs to be checked. And setting a preset condition according to the length of the reference character string, so that the reference character string of the vehicle identification code is verified by using the preset condition. Specifically, the number of characters included in a reference character string of the vehicle identification code is detected, and the character length of the reference character string is obtained. And judging whether the character length of the reference character string meets the length set in the preset condition or not. And recording a second detection result of the vehicle identification code as passing detection if the character length meets the preset condition, and recording the second detection result of the vehicle identification code as not passing detection if the character length does not meet the preset condition. For example, the vehicle VIN code consists of 17 characters, and the preset condition is: and if the length of the reference character string is 17 bits, recording a second detection result of the vehicle identification code as passing detection. If the length of the reference character string is not 17 bits, recording a second detection result of the vehicle identification code as a failed detection.

In one embodiment, as shown in fig. 4, detecting whether the image to be detected includes the vehicle identification code region image through the target detection model, and if so, extracting the vehicle identification code region image from the image to be detected, including the following steps:

s410, detecting the position data of the vehicle identification code area image in the image to be detected through the target detection model, and judging whether the vehicle identification code area image exists in the image to be detected or not;

s420, recording a third detection result of the vehicle identification code according to the judgment result of whether the vehicle identification code area image exists;

and S430, if the image to be detected has the vehicle identification code area image, extracting the image to be detected according to the position data to obtain the vehicle identification code area image.

Specifically, the position of the vehicle identification code area image in the image to be detected is detected, the confidence coefficient is calculated, and whether the vehicle identification code area image exists in the image to be detected is judged according to whether the confidence coefficient meets the preset threshold condition. Generally, the preset threshold of the confidence is set according to actual conditions, and may be 0.2 or 0.4, etc. And if the vehicle identification code regional image is detected to exist in the image to be detected, recording a third detection result of the vehicle identification code as passing detection, and if the vehicle identification code regional image is detected not to exist in the image to be detected, recording the third detection result of the vehicle identification code as not passing detection. Further, if the vehicle identification code area image is detected to exist in the image to be detected, the image to be detected is extracted or intercepted according to the position of the vehicle identification code area image in the image to be detected, and the vehicle identification code area image is obtained.

In this embodiment, the position data of the vehicle identification code region image in the image to be detected is detected by the target detection model, and whether the vehicle identification code region image exists in the image to be detected is determined. If the image exists, the image of the vehicle identification code area is extracted from the image to be detected, so that false detection caused by complex conditions in other background images in the image to be detected can be effectively avoided, a foundation is laid for detection work of the vehicle identification code, and the accuracy of vehicle identification code detection is improved.

In one embodiment, the recognition result of the vehicle identification code comprises the position of each character in the image of the vehicle identification code area, and the vehicle identification code is stored with a corresponding rubbing film picture in advance. As shown in fig. 5, after generating the character string corresponding to the vehicle identification code area image according to the recognition and segmentation result of the vehicle identification code, the method further includes the following steps:

s510, dividing the vehicle identification code area image into characters in the vehicle identification code according to the positions of the characters in the vehicle identification code area image;

s520, comparing each character obtained by segmentation with each character in the rubbing film picture, and judging whether each character obtained by segmentation is tampered;

and S530, recording a fourth detection result of the vehicle identification code according to the judgment result of whether each character obtained by segmentation is falsified.

In order to detect whether the characters in the vehicle identification code are tampered, the rubbing film picture corresponding to the vehicle identification code to be detected and each character in the rubbing film picture are prestored in the local of the first computer device or a server in communication connection with the first computer device. Specifically, each character in the vehicle identification code is identified through the character instance segmentation model on the vehicle identification code area image, and meanwhile, the position of each character in the vehicle identification code is obtained through target detection on the vehicle identification code area image, namely, the identification result of the vehicle identification code comprises the position of each character in the vehicle identification code area image. The vehicle identification code area image can be divided into characters according to the positions of the characters in the vehicle identification code area image, the divided characters are compared with the characters in the rubbing film picture from the aspects of character shapes, character positions, character intervals, character heights and the like, and whether the characters obtained by dividing the vehicle identification code area image are tampered or not is judged. And recording a fourth detection result of the vehicle identification code as a pass detection if detecting that the characters obtained by dividing the vehicle identification code area image are not tampered, and recording a fourth detection result of the vehicle identification code as a fail detection if detecting that the characters obtained by dividing the vehicle identification code area image are tampered.

In the embodiment, after the character string corresponding to the image of the vehicle identification code area is detected, whether the character in the vehicle identification code is tampered is further detected, so that the working efficiency of detecting the vehicle identification code is improved, and the accuracy of the detection work of the vehicle identification code is ensured.

In one embodiment, as shown in fig. 6, the generation of each character in the rubbing picture comprises the following steps:

s610, acquiring a rubbing film picture corresponding to the vehicle identification code;

s620, identifying the rubbing film picture through the character instance segmentation model to obtain each character in the rubbing film picture and the position of each character in the rubbing film picture;

s630, dividing the rubbing film picture into the characters in the rubbing film picture according to the positions of the characters in the rubbing film picture.

Specifically, a rubbing film picture corresponding to the vehicle identification code to be detected is obtained locally from the second computer device, or the rubbing film picture corresponding to the vehicle identification code to be detected is obtained from a server in communication connection with the second computer device. The rubbing film picture is identified by utilizing the character example segmentation model, each character in the rubbing film picture is identified, and simultaneously, each character in the rubbing film picture is subjected to target detection, so that the position of each character in the rubbing film picture is known. In order to detect whether each character of the vehicle identification code in the image to be detected is tampered, the rubbing film picture is divided into characters according to the position of each character in the rubbing film picture, and each character in the rubbing film picture is obtained.

In the embodiment, the rubbing film picture is identified through the character instance segmentation model, even if the rubbing film picture comprises continuous same characters, each character in the rubbing film picture and the position of each character can be accurately identified, so that the rubbing film picture is segmented according to the position of each character in the rubbing film picture, and each character in the rubbing film picture is obtained.

In one embodiment, the method further comprises: performing statistical analysis on the first detection result, the second detection result, the third detection result and the fourth detection result; when the first detection result, the second detection result, the third detection result and the fourth detection result are all detected to be passed, a detection result that the vehicle identification code is detected to be passed is generated; and when one or more of the first detection result, the second detection result, the third detection result and the fourth detection result are not passed, generating a detection result that the vehicle identification code is not passed.

Specifically, throughout the vehicle identification code detection: firstly, comparing a character string corresponding to a vehicle identification code area image with a reference character string of a vehicle identification code, and recording a first detection result of the vehicle identification code; recording a second detection result of the vehicle identification code by judging whether the character length of the reference character string meets a preset condition; recording a third detection result of the vehicle identification code by judging whether the image to be detected has the vehicle identification code area image; and comparing each character obtained by segmenting the image of the vehicle identification code region with each character in the rubbing film picture, and recording a fourth detection result of the vehicle identification code. And finally, determining the final detection result of the vehicle identification code through statistical analysis of the first detection result, the second detection result, the third detection result and the fourth detection result. When the first detection result, the second detection result, the third detection result and the fourth detection result are all detected to be passed, a detection result that the vehicle identification code is detected to be passed is generated; and when one or more of the first detection result, the second detection result, the third detection result and the fourth detection result are not passed, generating a detection result that the vehicle identification code is not passed.

Illustratively, corresponding flag bits are set for the first detection result, the second detection result, the third detection result and the fourth detection result, and different boolean values are set to indicate different detection results. If the character string corresponding to the vehicle identification code area image is consistent with the reference character string of the vehicle identification code, recording 1 at the first zone bit corresponding to the first detection result, and if the character string corresponding to the vehicle identification code area image is not consistent with the reference character string of the vehicle identification code, recording 0 at the first zone bit corresponding to the first detection result; if the character length meets the preset condition, recording 1 in a second zone bit corresponding to the second detection result, and if the character length does not meet the preset condition, recording 0 in a second zone bit corresponding to the second detection result; if the image to be detected exists in the vehicle identification code area image, recording 1 at a third zone bit corresponding to a third detection result, and if the image to be detected does not exist, recording 0 at the third zone bit corresponding to the third detection result; if the characters obtained by dividing the vehicle identification code area image are not falsified, 1 is recorded in the fourth flag bit corresponding to the fourth detection result, and if the characters are falsified, 0 is recorded in the fourth flag bit corresponding to the fourth detection result. When the first zone bit, the second zone bit, the third zone bit and the fourth zone bit all record 1, the image to be detected passes the final detection, and when any one of the first zone bit, the second zone bit, the third zone bit and the fourth zone bit records 0, the image to be detected does not pass the detection.

In one embodiment, as shown in fig. 7a, the recognizing and segmenting of the vehicle identification code region image is performed through a character instance segmentation model, and a character string corresponding to the vehicle identification code region image is generated according to the recognizing and segmenting result of the vehicle identification code region image, which includes the following steps:

s710, carrying out target detection on the vehicle identification code area image through a character instance segmentation model, and determining each character and the position of each character in the vehicle identification code area image;

s720, carrying out target classification on the vehicle identification code area image through the character instance segmentation model, and determining the category of each character in the vehicle identification code area image;

s730, performing pixel-level target segmentation on the vehicle identification code region image through the character instance segmentation model to obtain a plurality of pixel-level segmentation objects;

s740, selecting each character corresponding to the vehicle identification code from the multiple pixel level segmentation objects according to each character and each character type in the vehicle identification code area image;

and S750, sorting the characters according to the positions and the types of the characters in the vehicle identification code area image, and generating a character string corresponding to the vehicle identification code area image.

The method comprises the following steps of identifying a vehicle identification code region image by adopting a character instance segmentation model, wherein the character instance segmentation model needs to finish three things: character detection, character classification, and character segmentation on a pixel level hierarchy. Specifically, as shown in fig. 7b, the vehicle identification code region image is input to the input layer of the character instance segmentation model for some processing, and the output of the input layer is transmitted to the base network connected to the input layer. And extracting a Feature image (Feature Map) corresponding to the vehicle identification code area image through the base network. Feature images output by the base network are transmitted to a target detection candidate frame generation network and an ROI (region of Interest) Align layer, and the ROI Align layer is connected to the target detection candidate frame generation network. Two branches are connected behind the ROIAlign layer and respectively form a first branch and a second branch. Wherein the first branch comprises a first sub-branch and a second sub-branch. The first subbranch comprises a frame regression layer, and the first subbranch is used for carrying out target detection on the vehicle identification code area image and determining each character and the position of each character in the vehicle identification code area image. The second subbranch comprises a classifier, and the second subbranch is used for carrying out target classification on the vehicle identification code area image to determine the category of each character in the vehicle identification code area image. The second branch comprises a Mask layer (also called a Mask layer), and the pixel-level target segmentation is carried out on the vehicle identification code region image through the second branch to obtain a plurality of pixel-level segmentation objects. Since the pixel-level segmentation object includes each character constituting the vehicle identification code and other backgrounds, each character corresponding to the vehicle identification code is selected from the plurality of pixel-level segmentation objects according to the detected each character and the category of each character. If each character has its own position in the vehicle identification code region image, the detected characters are sorted according to the position and type of each character in the vehicle identification code region image, and a character string corresponding to the vehicle identification code region image is generated, i.e., the generated character string is the vehicle identification code detected from the image to be detected.

In the embodiment, the character detection, the character classification and the character segmentation on the pixel level layer are respectively carried out on the vehicle identification code region image through the character instance segmentation model, so that the accurate identification of continuous same characters in the image to be detected of the vehicle identification code is realized, and the accuracy of the vehicle identification code detection is improved.

In one embodiment, the structure of the character instance segmentation model is shown in FIG. 7b, and the character instance segmentation model calls Resnet-101 as the base network, using an FPN (feature metadata networks) structure. And the character instance segmentation model uses a semantic segmentation function. Specific hyper-parameters may be set as: the initial learning rate BASE _ LR is 0.01, the learning rate DECAY SIZE WEIGHT _ DECAY is 0.0001, the learning rate DECAY strategy is multistep, the maximum picture SIZE MAX _ SIZE _ trans is 800, and the minimum picture SIZE MIN _ SIZE _ trans is 500. Specifically, the generation of the character instance segmentation model comprises the following steps: acquiring a sample set of the vehicle identification code area image; and training a character example segmentation model according to each vehicle identification code area image, the position of each character in each vehicle identification code area image and the category corresponding to each character in each vehicle identification code area image.

The sample set of the vehicle identification code area images comprises a plurality of vehicle identification code area images, and the positions and the types of the characters in the vehicle identification code area images are labeled. For example, the vehicle identification code area image may be labeled by an open-source label tool. The vehicle VIN code has 34 characters of 0-9, A-N, P and R-Z, and the class (class) can be represented by 0-33. Since the character instance segmentation model needs to identify each character of the same category, the same character appearing in the vehicle identification code region image needs to be labeled respectively during labeling.

Specifically, the sample set of the vehicle identification code area image is obtained as the training set in various ways, for example, the second computer device obtains the sample set of the vehicle identification code area image from a server storing the sample set of the vehicle identification code area image through a wired connection way or a wireless connection way. Alternatively, the vehicle identification code region image sample set is stored locally in the second computer device, and the second computer device obtains the vehicle identification code region image sample set locally. And the position and the category of each character are marked on each image of the vehicle identification code area in the sample set. And the second computer equipment trains the character instance segmentation model by adopting each vehicle identification code area image, the position of each character in each vehicle identification code area image and the category corresponding to each character in each vehicle identification code area image. And according to the difference between the result predicted by the character instance segmentation model and the label, adjusting the model parameters of the character instance segmentation model and continuing training until the training stopping condition is met. Wherein the training stop condition is a condition for ending the model training. The training stopping condition may be that a preset number of iterations is reached, or that the performance index of the character instance segmentation model after the model parameters are adjusted reaches a preset index.

In one embodiment, as shown in FIG. 8, the generation of the sample set of vehicle identification code region images includes the following steps:

and S810, acquiring a sample set of the original vehicle identification code image.

Wherein the sample set of original vehicle identification code images includes a plurality of original vehicle identification code images. The original vehicle identification code image refers to an image for passing through object detection and intercepting or extracting it to obtain a vehicle identification code area image. Specifically, the sample set of the original vehicle identification code image is obtained in various ways, such as the second computer device obtaining the sample set of the original vehicle identification code image from a server storing the sample set of the original vehicle identification code image through a wired connection or a wireless connection. Alternatively, the sample set of the original vehicle identification code image is first stored locally on the second computer device, and the second computer device obtains the sample set of the original vehicle identification code image locally.

And S820, acquiring the position data of the vehicle identification code area in each original vehicle identification code image.

Specifically, the vehicle identification code region in each original vehicle identification code image is detected by an object detection algorithm. The target detection algorithm is an algorithm for scanning an original vehicle identification code image by using a sliding window to find a vehicle identification code area contained in the image and calculate the vehicle identification code area. The output of the target detection algorithm includes the coordinates of the vehicle identification code zone circumscribed rectangle or the vehicle identification code zone circumscribed rectangle frame in the original vehicle identification code image. So that the position data of the vehicle identification code region in each of the original vehicle identification code images can be acquired.

And S830, extracting each vehicle identification code area image from each original vehicle identification code image according to the position data of the vehicle identification code area.

Specifically, given the position data of the vehicle identification code region in each original vehicle identification code image, the vehicle identification code region in each original vehicle identification code image can be extracted according to the position data of each vehicle identification code region, so as to obtain each vehicle identification code region image. It is understood that the extracted respective vehicle identification code region images may have the same size.

And S840, marking the position and the type of each character in each vehicle identification code area image by adopting a character edge-drawing method.

Specifically, for each extracted vehicle identification code area image, the position and the category of each character in each vehicle identification code area image are labeled by a character edge tracing method. For example, one vehicle identification code region image includes 17 characters, each character corresponds to a respective category, if consecutive identical characters appearing in the vehicle identification code region image are also respectively labeled, and when the labeling is completed, a corresponding labeling file, such as a JSON file, can be generated.

And S850, performing data expansion operation on the marked vehicle identification code area images.

The data expansion refers to a means for increasing the number of training sets by image deformation or noise increase. The data augmentation operation includes at least one of a layout transformation, a bending transformation, a rotation, and a translation. Specifically, because the labeling work of the vehicle identification code is relatively time-consuming and the labeling cost is relatively high, the extracted vehicle identification code region image can be subjected to layout transformation, bending transformation, rotation, translation and other operations, and one vehicle identification code region image is expanded into a preset number of different forms of pictures, for example, the original one picture is expanded into 10 different forms of pictures.

Wherein the bending transformation operation is specifically: acquiring coordinates of four vertexes of the vehicle identification code area image and the position of a lens center in the image, wherein the lens center is the center of the vehicle identification code area image; the image is subjected to a warping transformation by a specific function. Further, border crossing protection is carried out while bending transformation is carried out, if characters in the image exceed the image range in the process of bending transformation, incomplete characters are caused, and in order to avoid bad influence on training of a subsequent character instance segmentation model, the extended image beyond the range is deleted.

Generally, the characters in the vehicle identification code are arranged in a single row or double rows, and for the case that the vehicle identification code includes 17 characters, the statistical analysis of the picture data of a large number of double rows of vehicle identification codes finds that the first row of the double rows of vehicle identification codes is usually 9 characters, and the second row is 8 characters. The layout transformation operation is illustrated below: and intercepting any partial area which does not contain the vehicle identification code characters in the vehicle identification code area image, and adjusting the partial area to a preset size to be used as the background of the new expanded image. The external rectangular frame of each character in the vehicle identification code area image can be obtained through manual labeling or a target detection algorithm, two parts of images of the vehicle identification code in a single row are intercepted according to the coordinates of the right lower vertex of the 9 th character and the coordinates of the left upper vertex of the next character, after proper size adjustment, the images are pasted on the background of a new expanded image, and the vehicle identification code area image with the vehicle identification codes in double rows is obtained. And if the incomplete characters exist, deleting the corresponding extended image.

And S860, generating a sample set of the vehicle identification code area images according to the marked vehicle identification code area images and the vehicle identification code area images obtained through data expansion operation.

Specifically, each vehicle identification code area image is extracted from each original vehicle identification code image, each extracted vehicle identification code area image is labeled, and each labeled vehicle identification code area image can be used as a training set of a character instance segmentation model. The images of the respective vehicle identification code regions obtained by the data expansion operation can also be used as a training set of the character instance segmentation model.

In the embodiment, the same characters appearing in the vehicle identification code area image are respectively marked, so that the trained character instance segmentation model can identify continuous same characters, the sample set of the vehicle identification code area image is more balanced by improving the extended sample set, the model has stronger generalization capability, and various types of vehicle identification codes can be detected more accurately.

In one embodiment, the present application provides a method for detecting a vehicle identification code, the method comprising the steps of:

s902, acquiring a sample set of original vehicle identification code images, wherein the sample set of the original vehicle identification code images comprises a plurality of original vehicle identification code images.

And S904, acquiring the position data of the vehicle identification code area in each original vehicle identification code image.

And S906, extracting each vehicle identification code area image from each original vehicle identification code image according to the position data of the vehicle identification code area.

And S908, marking the position and the type of each character in each vehicle identification code area image by adopting a character edge-drawing method.

S910, performing data expansion operation on the marked vehicle identification code area images.

Wherein the data expansion operation comprises at least one of layout transformation, bending transformation, rotation and translation.

S912, generating a sample set of the vehicle identification code area images according to the marked vehicle identification code area images and the vehicle identification code area images obtained through data expansion operation.

S914, training a character example segmentation model according to each vehicle identification code area image in the sample set, the position of each character in each vehicle identification code area image and the corresponding category of each character in each vehicle identification code area image.

S916, an image to be detected of the vehicle identification code and a reference character string of the vehicle identification code are obtained, and the image to be detected comprises a vehicle identification code area image.

S918, acquiring the character length of the reference character string of the vehicle identification code.

S920, judging whether the character length of the reference character string meets a preset condition.

And S922, recording 1 in the first zone bit corresponding to the character length if the character length meets the judgment result of the preset condition, and recording 0 in the first zone bit of the character length if the character length does not meet the judgment result of the preset condition, and storing the related picture.

And S924, if so, detecting the position data of the vehicle identification code region image in the image to be detected through the target detection model, and judging whether the vehicle identification code region image exists in the image to be detected.

If the image exists, recording 1 in the second zone bit corresponding to the vehicle identification code zone image, and if the image does not exist, recording 0 in the second zone bit corresponding to the vehicle identification code zone image, and storing the related picture.

And S926, if the image exists, extracting the image to be detected according to the position data to obtain a vehicle identification code area image.

And S928, recognizing and dividing the vehicle identification code area image through the character instance division model, and generating a character string corresponding to the vehicle identification code area image according to the recognition and division result of the vehicle identification code area image.

The recognition result of the vehicle identification code comprises the position of each character in the image of the vehicle identification code area, and the vehicle identification code is pre-stored with a corresponding rubbing film picture.

S930, comparing the character string corresponding to the vehicle identification code area image with the reference character string of the vehicle identification code.

If the two are consistent, recording 1 in the third flag bit corresponding to the comparison result, and if the two are not consistent, recording 0 in the third flag bit corresponding to the comparison result, and storing the related pictures.

And S932, dividing the vehicle identification code area image into the characters in the vehicle identification code according to the positions of the characters in the vehicle identification code area image.

And S934, comparing each character obtained by segmentation with each character in the rubbing film picture, and judging whether each character obtained by segmentation is tampered.

If the picture is not tampered, recording 1 in the fourth flag bit corresponding to the tampered state, and if the picture is tampered, recording 0 in the fourth flag bit corresponding to the tampered state, and storing the related picture.

And S936, if the first zone bit, the second zone bit, the third zone bit and the fourth zone bit are all recorded with 1, detecting the to-be-detected image of the vehicle identification code.

And S938, if any one of the first zone bit, the second zone bit, the third zone bit and the fourth zone bit records 0, the image to be detected of the vehicle identification code does not pass the detection.

And S940, inquiring the reason why the image to be detected does not pass the detection and the corresponding picture according to the flag bit recorded as 0.

It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above embodiments may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the sub-steps or the stages of other steps.

In one embodiment, the present application provides a vehicle identification code detection apparatus 900, as shown in fig. 9, comprising:

the acquiring module 910 is configured to acquire an image to be detected of a vehicle identification code and a reference character string of the vehicle identification code, where the image to be detected includes a vehicle identification code region image;

a detection module 920, configured to detect whether a vehicle identification code region image exists in the image to be detected through a target detection model, and if so, extract the vehicle identification code region image from the image to be detected;

the recognition module 930 is configured to perform recognition and segmentation on the vehicle identification code region image through a character instance segmentation model, and generate a character string corresponding to the vehicle identification code region image according to a recognition and segmentation result of the vehicle identification code region image;

a comparison module 940, configured to compare a character string corresponding to the vehicle identification code region image with a reference character string of the vehicle identification code;

the recording module 950 is configured to record the first detection result of the vehicle identification code according to the comparison result.

For specific limitations of the vehicle identification code detection device, reference may be made to the above limitations of the vehicle identification code detection method, which are not described herein again. The modules in the vehicle identification code detection device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of detecting a vehicle identification code. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps of the above embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the method steps of the above-mentioned embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.