阅读说明：本技术 用于识别车辆牌照的方法和装置 (Method and device for recognizing license plate of vehicle ) 是由 维奥莱特·斯内尔 于 2019-07-11 设计创作，主要内容包括：本发明提供了一种用于识别车辆的牌照的装置,该装置包括用于读入光学传感器的周围环境的图像的接口,该图像源自该光学传感器对具有可检测牌照的至少所述车辆进行拍照。该装置进一步包括用于使用卷积神经网络分析该图像的单元,该卷积神经网络具有至少两个分离的符号标识分支,这些分离的符号标识分支中的每一个被配置用于标识该牌照的若干符号之一和/或该卷积神经网络具有与这些符号标识分支分离的国家标识分支,该国家标识分支被配置用于标识已发放该牌照的国家。最后,该装置包括用于输出至少两个所标识的符号或所标识的国家,以便识别该车辆的牌照的单元。(The invention provides a device for identifying a license plate of a vehicle, the device comprising an interface for reading in an image of the surroundings of an optical sensor, the image originating from the optical sensor taking a picture of at least said vehicle with a detectable license plate. The apparatus further includes means for analyzing the image using a convolutional neural network having at least two separate symbol-identifying branches, each of the separate symbol-identifying branches configured to identify one of the symbols of the license plate and/or a country-identifying branch separate from the symbol-identifying branches, the country-identifying branch configured to identify a country in which the license plate has been issued. Finally, the device comprises means for outputting at least two identified symbols or identified countries for recognizing the license plate of the vehicle.)

1. A method (300) for identifying a license plate (105) of a vehicle (110), the method (300) comprising the steps of:

-reading in (310) an image (135) of the surroundings of the optical sensor (130), which image (135) originates from the optical sensor (130) taking a picture of at least said vehicle (110) with the detectable license plate (105);

-analyzing (320) the image (135) using a convolutional neural network (150), the convolutional neural network (150) having at least two separate symbol-identifying branches (220a, 220b), each of the separate symbol-identifying branches (220a, 220b) being configured to identify one of several symbols (155) of the license plate (105) and/or the convolutional neural network (150) having a country-identifying branch (225) separate from the symbol-identifying branches (220a, 220b), the country-identifying branch (225) being configured to identify a country (160) that has issued the license plate (105); and

-outputting (330) the at least two identified symbols (155) and/or the identified country (160) for identifying the license plate (105) of the vehicle (110).

2. The method (300) of claim 1, wherein in the analyzing step (320), a convolutional neural network (150) is used, which has a full connectivity layer (230) in the symbol-identifying branches (220a, 220 b).

3. The method (300) according to one of the preceding claims, characterized in that in the analyzing step (320) a convolutional neural network (150) is used, which has an output layer (240) in the symbol identification branches (220a, 220b), which output layer (240) has a number of neurons which is identical to the number of identifiable distinguishable types of symbols (155) on the license plate (105), in particular the output layer (240) has 37 neurons.

4. The method (300) according to one of the preceding claims 2 and 3, characterized in that in the analysis step (320) a convolutional neural network (150) is used, which has the fully-connected layer (230) as input to the output layer (240) in the symbol-identifying branches (220a, 220 b).

5. The method (300) according to one of the preceding claims, characterized in that in the analyzing step (320) a convolutional neural network (150) is used, which has at least three separate symbol identification branches (220a, 220b), each of the separate symbol identification branches (220a, 220b) being configured for identifying one of several symbols (155) of the license plate (105), and in the outputting step (330) the at least three identified symbols (155) are output for identifying the license plate (105) of the vehicle (110).

6. The method (300) according to one of the preceding claims, wherein in the analyzing step (320) a convolutional neural network (150) is used, having a plurality of convolutional layers (200), in particular having 3 to 20, preferably 5 convolutional layers, which are arranged in sequence, the plurality of convolutional layers (200) being used for processing the image (135) to obtain processed information (205, 215), and wherein the convolutional neural network (150) is configured for feeding the processed information (205, 215) into the separate symbol identification branches (220a, 220b) and/or the country identification branch (225).

7. The method (300) according to one of the preceding claims, characterized in that in the analyzing step (320) a convolutional neural network (150) is used, which has at least one shared full connectivity layer (210) as input to the separate symbol identification branches (220a, 220b) and/or the country identification branch (225).

8. A method (400) for training a convolutional neural network (150) for use in a method (300) for identifying a license plate (105) of a vehicle (110), the method (400) comprising the steps of:

-reading (410) an image (135) of the surroundings of the optical sensor (130), the image (135) taking a picture of at least the vehicle (110) with the detectable license plate (105);

-providing (420) a convolutional neural network (150), the convolutional neural network (150) having at least two separate symbol identification branches (220a, 220b), each of the separate symbol identification branches (220a, 220b) being configured to identify one of a number of symbols (155) of the license plate (105) and/or the convolutional neural network (150) having a country identification branch (225) separate from the symbol identification branches (220a, 220b), the country identification branch (225) being configured to identify a country (160) to which the license plate (105) has been issued; and

-training (430) the convolutional neural network (150) using the image (135) read in.

9. An apparatus (100) configured for controlling and/or performing the steps of one of the methods (300, 400) according to one of the preceding claims in a corresponding unit (140, 145, 165) of the apparatus (100).

10. A computer program configured for controlling and/or executing the steps of a method (300, 400) according to one of the preceding claims if the computer program runs on a respectively configured apparatus (100).

11. A machine readable carrier having stored thereon a computer program according to claim 10.

Technical Field

The present invention relates to a method and apparatus for identifying a vehicle license plate, and a computer program.

Background

Most current methods followed for automatic license plate recognition (ANPR) consist of a pipeline that first locates the area containing the character, then divides it into separate character boxes (a process known as segmentation), followed by independent recognition of the letter or number represented at each location. Some post-processing logic may then attempt to conform the results to the grammatical rules of this country, but need to identify the country correctly. The pipeline is prone to errors at each stage of the pipeline, and cumulative effects often result in low recognition accuracy.

Disclosure of Invention

Based on this state of the art, the present invention discloses a novel method and apparatus for recognizing a license plate of a vehicle, and a computer program. Other aspects of the invention are provided by advantageous embodiments.

Disclosed is a method for recognizing a license plate of a vehicle, the method including the steps of:

-reading in an image of the surroundings of an optical sensor, which image originates from the optical sensor taking a picture of at least said vehicle with a detectable license plate;

-analyzing the image using a convolutional neural network, the convolutional neural network having at least two separate symbol identification branches, each of the separate symbol identification branches being configured to identify one of the symbols of the license plate and/or the convolutional neural network having country identification branches separate from the symbol identification branches, the country identification branches being configured to identify the country in which the license plate has been issued; and

-outputting the at least two identified symbols and/or the identified country for identifying the license plate of the vehicle.

According to the methods disclosed herein, an image may be considered an output signal of a camera or an optical sensor. A symbol identification branch may be considered to be a layer or a set of layers that is separated from another layer or set of layers to identify one of several symbols of a license plate. The separation of layers or branches can be understood as follows: no information from one layer or branch is transferred to another layer of a different branch. In this description, a symbol may be understood as a number or a character or a space or at least one tax label that may be part of a registration string depicted on a vehicle license plate. Thus, the identification of a vehicle license plate may be understood as identifying a set of symbols depicted on the vehicle license plate to uniquely identify the vehicle. Additionally or alternatively, in the analyzing step, a convolutional neural network is used, the convolutional neural network having a country identification branch separate from the symbol identification branch, the country identification branch being configured to identify the country from which the license plate has been issued. The use of a specially configured country identification branch provides the following advantages: the most specific design of which country the license plate originates from may be used to facilitate identification of that country. Such identification may be accomplished by identifying a particular marking (e.g., indicia or code) on the license plate and/or by identifying the font, spacing, and syntax of the symbols depicted on the license plate. Thus, all of the graphical information may be utilized to identify the corresponding country on the license plate, as well as any country-specific format or number of country-specific symbols expected during recognition of the license plate. Additionally, the country identification branch may be configured to identify which country the license plate originates from by: the symbol structure, style, spacing between symbols, etc. of the registration codes are analyzed to identify which country the license plate originates from using differences in the registered license plate that relate to the different symbols depicted on the license plates of different countries and/or the fonts, spacing, and syntax of the symbols. The use of separate country identification branches also indirectly further improves the recognition of license plate strings.

The present invention is based on the following findings: the use of a convolutional neural network directly on the entire image originating from the optical sensor significantly improves the quality and speed of recognition of the vehicle license plate. The use of at least two separate symbol identification branches, in particular one of several symbols for identifying a license plate, provides a high flexibility in identifying different license plates without being constrained by a pre-segmentation of the area in the image identified as containing the license plate image. Thus, using the methods disclosed herein provides the following advantages: pre-identification of the exact position of the license plate in the image and pre-segmentation of this area in the image for identifying different numbers or digits in the segment is omitted, which in turn facilitates pre-processing of the image to identify the license plate of the vehicle. Instead, only an image (or the like) of the approximate neighborhood of the license plate needs to be fed into a trained convolutional neural network, which then provides at least two identified symbols in order to recognize the license plate of the vehicle.

According to a particular embodiment of the invention, in the analysis step, a convolutional neural network is used, which has at least a fully connected layer in the sign-identifying branch. In a fully connected layer, each neuron in one layer is connected to each neuron in another layer. Such an embodiment of the invention provides the following advantages: the identification of the symbols may be performed with high accuracy, so that, for example, soiling on the license plate or damage to the license plate does not hinder or prevent the method disclosed herein from identifying the correct symbols at the corresponding positions.

Furthermore, in accordance with another embodiment of the present invention, in the analyzing step, a convolutional neural network is used, the convolutional neural network having an output layer in the symbol identification branch, the output layer having a plurality of neurons equivalent to the plurality of identifiable distinguishable types of symbols on the license plate, in particular the output layer having 37 neurons. These recognizable symbols may be, for example, numbers, space symbols, or characters of an alphabet that may be depicted on the license plate to form a unique registration string or sequence of symbols on the license plate for identifying the corresponding vehicle. In particular, 10 digits, space symbols and 26 alphabetic characters may form the basis of a registration string for license plates, so that in most countries where the western alphabet is utilized, identification with 37 symbols is sufficient.

According to a further embodiment of the invention, in the analysis step a convolutional neural network is used, which has a fully connected layer as input to the output layer in the sign-identifying branch. In particular, combining fully-connected layers as inputs to an output layer having a plurality of neurons equivalent to a plurality of identifiable symbols on a license plate provides greater flexibility and greater accuracy in identifying the correct symbol in each symbol-identifying branch.

A method according to any one of the preceding methods, characterized in that in the analyzing step a convolutional neural network is used, which convolutional neural network has at least three separate symbol identification branches, each of which is configured to identify one of several symbols of the license plate, and in the outputting step the at least three identified symbols are output in order to identify the license plate of the vehicle. To provide a one-step identification or recognition of the license plate, more than two separate symbol identification branches may be used, for example at least three separate symbol identification branches. More specifically, if the convolutional neural network is trained on a license plate of a particular country so that it can know how many positions there can be for a registration string or a unique symbol sequence on the license plate, a corresponding number of separate symbol identification branches can be used in the convolutional neural network. Furthermore, the neural network may be further configured to output space symbols to enable filling if the number of symbols of the license plate shown in the image (these symbols may be for the license plate of the corresponding country where the license plate is to be recognized) is not full.

According to a further embodiment of the invention, in the analyzing step, a convolutional neural network is used, which has at least one shared fully-connected layer as input to the separate symbolic identification branch and/or the country identification branch. Such an embodiment of the invention provides the following advantages: the information relating to all locations and hence all branches can be collated according to all spatial locations within the convolutional layer and otherwise processed in an overall manner. It enables the exchange of information between branches by providing shared inputs to the branches, the composition of which is affected by all the branches during training.

In order to provide sufficient information in the convolutional neural network to identify or recognize the license plate, an advantageous embodiment of the invention may be used, in which a convolutional neural network is used in the analyzing step, the convolutional neural network having a plurality of convolutional layers. In particular, the convolutional neural network may have 3 to 20, preferably 5 convolutional layers. The plurality of convolutional layers may be used to process the image to obtain processed information, wherein the convolutional neural network is further configured to feed the processed information into separate sign-identifying branches and/or country-identifying branches. Such an embodiment of the invention provides the following advantages: symbols of various sizes are identified with sufficient accuracy while limiting the computational cost of both training and deployment.

To achieve the above advantages, the convolutional neural network must be trained before feeding the images. Such training may be performed, for example, in a laboratory environment using pre-recorded images of vehicles with corresponding detectable license plates. Thus, according to another embodiment of the invention, there is disclosed a method for training a convolutional neural network for use in a method for identifying a license plate of a vehicle, the method comprising the steps of:

-reading in an image of the surroundings of an optical sensor, the image originating from the optical sensor taking a picture of at least the vehicle with the detectable license plate;

-providing a convolutional neural network having at least two separate symbol identification branches, each of the separate symbol identification branches being configured to identify one of the symbols of the license plate and/or the convolutional neural network having country identification branches separate from the symbol identification branches, the country identification branches being configured to identify the country in which the license plate has been issued; and

-training the convolutional neural network using the image read in.

This embodiment of the invention also facilitates the achievement of the above-mentioned advantages.

The methods disclosed herein may be implemented, for example, in software or hardware, or in a hybrid form of software and hardware (e.g., in a control unit).

The method presented herein also provides an apparatus or device designed to perform, control and/or implement the steps of the method variants presented herein in the respective facilities or units. Furthermore, by this embodiment of the invention in the form of a device, the potential objects of the invention can be achieved quickly and efficiently.

To this end, the device may have: at least one arithmetic unit for processing signals or data; at least one memory unit for storing signals or data; at least one interface to a sensor or actuator for reading a sensor signal from the sensor or for outputting a data or control signal to the sensor, actuator; and/or at least one communication interface for reading or outputting data embedded in a communication protocol. The arithmetic unit may be, for example, a signal processor, a microcontroller, etc., wherein the memory unit may be a flash memory, an EEPROM or a magnetic memory unit. The communication interface may be configured to read or output data wirelessly and/or via a cable, wherein the communication interface may read or output wired data, e.g. read this data electrically or optically from or output in a respective data transmission line.

In this case, the device may be understood as meaning an electrical device which processes sensor signals and outputs control signals and/or data signals in dependence on these sensor signals. The device may have an interface that may be formed in hardware and/or software. In the case of a hardware-based embodiment, the interface may be part of a so-called system ASIC, for example, which contains the various functions of the device. However, the interface may also be its own integrated circuit or be at least partly composed of discrete components. In a software embodiment, the interface may be a software module, for example, residing on a microcontroller, among other software modules. The device is configured for controlling and/or performing the steps of one of the methods according to any of the preceding in a corresponding unit of the device.

The invention also has the following advantages: a computer program product or a computer program is provided with a program code, which can be stored on a machine-readable carrier or storage medium, such as a semiconductor memory, a hard disk memory or an optical memory, and which is used, in particular when executed on a computer or a device, to perform, implement and/or control the steps of one of the methods according to one of the above-mentioned embodiments.

Drawings

Embodiments of the methods presented herein are illustrated in the drawings and explained in more detail in the following description. In the drawings:

fig. 1 shows a schematic view of a scenario utilizing an embodiment of the invention as a device for recognizing a license plate of a vehicle;

FIG. 2 is a block diagram of an example of a convolutional neural network that may be used in a unit for performing an analysis according to FIG. 1;

fig. 3 is a flow chart of a method for identifying a license plate of a vehicle according to an embodiment of the present invention; and is

Fig. 4 is a flow chart of a method for training a convolutional neural network for use in a method for identifying a license plate of a vehicle.

Detailed Description

In the following description of advantageous embodiments of the invention, the same or similar reference numerals are used for the same or similar acting elements shown in the various figures, wherein a repeated description of these elements is omitted.

Fig. 1 shows a schematic view of a scenario in which an embodiment of the invention is utilized as an apparatus 100 for identifying a license plate 105 of a vehicle 110. For example, vehicle 110 is traveling on road 115, and control pillars 120 are placed alongside the road, for example (but not mandatory) to measure the speed of vehicle 110 by means of radar sensor 125 or to check whether a specific amount of the required road charge for vehicle 110 has been paid or is about to be paid. However, the vehicle 110 is in the embodiment according to fig. 1 shown as a passenger car (this is not mandatory), so that the vehicle 110 may also be a truck, bus, motorcycle or the like, all having a unique license plate 105 in front of or at least behind. In order to identify the vehicle 110 by means of automatic recognition of the license plate 105, the control column 120 comprises a camera 130 (which may also be denoted as an optical sensor) which captures an image 135 of the camera 130, for example of the surroundings in which the vehicle 110 is travelling on the road 115, in case a ticket is issued or a required road charge is paid for due to a violation of a speed limit. In addition, under normal circumstances, license plate 105 is also detectable, which is required according to the traffic-related legal regulations of most countries.

The image 135 captured by the camera 130 is then forwarded to a read-in interface 140 of the apparatus 100 for use in identifying the license plate 105 of the vehicle 110. From this read-in interface 140 the image 135 is transferred to a unit 145 for analysis, wherein the image 135 is fed into a convolutional neural network 150, which is explained in more detail in the subsequent part of the description. The convolutional neural network 150 is configured to identify the country 160 in which the symbol 155 and/or vehicle 110 is licensed. Thus, the convolutional neural network 150 is configured to output the parameters of the identified symbol 155 and/or country of release 160, which are referenced in the box, with only the reference numeral of the symbol 155 representing the convolutional neural network 150 on the right hand side of fig. 1. These parameters of the identification symbol 155 and the country of issuance 160 are then output from the unit for analysis 145 to an output unit 165, which is then configured, for example but not compulsorily, to combine the individual symbols 155 and/or the code representing the country 160 to which the vehicle 110 is licensed, so that the complete registration string 170 (or symbol string) on the license plate 105 is recognized. The registration string 170 written on the license plate may then be transmitted downstream along with information about the identified country, which is not necessarily part of the registration string. This registration string 170 or symbol string on license plate 105 is then e.g. stored or transmitted to a separate unit 180 where e.g. identification of the vehicle owner or registration owner is performed, in order to deliver a ticket for violation of a speed limit or in order to charge a required road charge from the corresponding person.

Furthermore, the sliding window approach is very slow and requires training data with not only the labels of the correctly registered strings but also the label position of each character in order to train the single character classifier. Modeling the sequence with a generative HMM and selecting the best code with a naive bayes classifier is significantly less accurate than a discriminative trained neural net. The language model embodied by the HMM must accommodate the distribution of grammatical patterns for each state. Only the syntax is considered when attempting to determine the publication status.

Thus, starting from an image captured by a traffic monitoring camera showing a cropped area around the license plate 105 of the vehicle 110, the purpose of the method may be referred to, for example, for determining the sequence of letters and numbers (symbols 155) that make up the registration string 170. By using the method disclosed herein, the recognition accuracy in the presence of variations in the size and position of the license plate within the cutting area, its rotation/tilt, contrast, focus, and the presence of dirt, small obstacles, or physical plate damage may be improved. Further, the registered country can be determined with high accuracy, which enables accurate identification of a license plate that conforms to the license plate syntax rules of the relevant jurisdictions while distinguishing ambiguous cases such as 0 from 'O' or 1 from 'I'. In addition, the limited processing power of the embedded device may also be accommodated to allow near real-time execution.

Fig. 2 shows a block diagram of an example of a convolutional neural network 150 according to fig. 1 that can be used in the unit 145 for performing an analysis. As can be seen from fig. 2, an image 135, for example represented by digital image data, is fed into a convolutional neural network 150. The image 135 may, for example, be a matrix comprising 200 x 80 pixels, each pixel being captured in a single color. The image 135 is then fed, for example, into a plurality of convolutional layers with linear rectification activation and maximum pooling, then into a block 200 of fully connected layers. The number of convolutional layers may for example be between 3 and 7, preferably 5, which provides a good trade-off between the accuracy of identifying license plate 105 and the tolerable digital workload in order to obtain a corresponding result quickly. For example, if the block 200 of convolutional layers includes 5 convolutional layers, the respective layers may be configured as follows:

the 1 st convolutional layer has 32 5 × 5 kernels, and 2 × 2 max pooling

The 2 nd convolutional layer has 64 3 × 3 kernels, and 2 × 2 max pooling

-the 3 rd convolutional layer has 128 3 x 3 kernels, and 2 x 2 max pooling

The 4 th convolutional layer has 512 cores of 1 × 1 without maximum pooling

The 5 th convolutional layer has 64 1 × 1 kernels, and a maximum pooling of 1 × 2 (vertical only)

The dimensions of the fully connected layer are indicated in fig. 2.

All layers belonging to the exemplary block 200 use a linear rectifying activation function (with leakage ReLU) with a small negative slope. The 5 th convolutional layer may have a core of 1 × 1 or 3 × 3.

The output signals 205 of the block 200 of convolutional layers (which may be considered as a matrix of 23 x 5 dots/pixels in 64 channels Ch, for example), which may be considered as processed information, are then fed into a shared fully-connected layer 210 (again having linear rectifying activity, for example).

What can be considered as processed information 215 or signals sharing the fully connected layer 210 are then fed into separate symbol identification branches 220a, 220b and country identification branch 225. In figure 2, only 2 symbol identification branches 220 are disclosed for simplicity, however, those skilled in the art will readily appreciate that many more such symbol identification branches 220 may be implemented, preferably as many symbol identification branches 220 as are identified in the longest possible string 170 to be stamped on or implemented in license plate 105. In fig. 2, the symbol identification branch 220a represents an identification branch of a first symbol of the registration character string 170, and the second identification branch 220b represents an identification branch of an eighth symbol of the registration character string 170; for simplicity, the symbolic identification branch between branches 220a and 220b is not shown. Thus, the configuration shown in fig. 2 allows identification of 8 symbols in the registration string 170. It should be noted, however, that convolutional neural network 150 may have more or less than 8 symbolic identification branches 220. However, convolutional neural networks are configured to cover a set of countries and have as many sign branches as the longest possible registration in any country they are aware of. Any registrations that are shorter than the longest possible registration (because a particular registration is shorter or all registrations in a country are shorter) are filled with, for example, spaces.

The processed information 215 is, for example, a signal having 2048 components. For example, since the signal or processed information 215 has 2048 components, the shared fully-connected layer 210 may also be considered to have 2048 nodes, each of which provides one of the components of the processed information 215 (respectively, the signal).

Each of the symbol-identifying branches 220a and 220b are separated from each other such that, for example, there is no information that can be passed from one branch 220 to another branch 220. Each of the branches 220 is configured to identify one single symbol 155 in the number 170 of license plate 105 at a corresponding location in registration string 170. To accomplish this task, the symbolic identification branch 220 includes a fully-connected layer 230 that accepts the processed information 215 and outputs a signal 235 having, for example, 256 components, such that the fully-connected layer 230 can be considered to have 256 output nodes. The signal 235 of the symbol identification branch 220 is then fed into an output layer 240 of the branch 220, which then outputs the corresponding identified symbol 155 at the corresponding position in the registration string 170, which is analyzed by the corresponding symbol identification branch 220, e.g. via a soft-max function. More specifically, the identified symbol 155 is considered a set of predictions, i.e., how likely this symbol is for each of the 37 options. The most likely symbol is typically selected as the answer, referred to in this specification as the "identified symbol" 155, but other ways of handling 37 numbers are possible.

According to fig. 2, symbol 155 may be one of a predefined number of symbols (e.g., 37 symbols) representing possible symbols at any location in registration string 170. The number 37 is obtained, for example, by a 10 digit number, 26 alphabetic characters, and 1 space symbol that can be found in the corresponding position of the registration string 170.

In addition, the processed information 215 may be fed into the country identification branch 225 in order to identify the particular country to which the license plate 105 is authorized. Still in the country identification branch 225, at least one fully connected layer 245 for identifying a country is provided, which is fed by the processed information 215. Fully connected layer 245 outputs, for example, a signal 250 having 128 components which is then accepted by output layer 255 for outputting a code representing the country 160 to which license plate 105 is authorized. More specifically, the output 160 is not a direct flag or country label, but rather a set of probabilities, one for each possible country. Also, in this specification, the symbol that is generally the most likely will be selected and named country 160, but it is also possible to set a threshold below which the system outputs the answer "unknown". Depending on the training of the convolutional neural network 150, the output layer 255 for the output countries 160 may be configured such that a predefined number N of countries (denoted as N countries in fig. 2) may be identified.

In summary, the method disclosed herein, for example as an embodiment of the present invention, uses a single convolutional neural network 150 having multiple outputs to output an identified symbol 155 and, optionally, a code 160 indicating the country in which license plate 105 is authorized. Convolutional neural network 150 is used to simultaneously read number plate 105 (regardless of where it appears within the cropped area) and determine its registered country. The first fully connected layer is, for example, generic, while the following fully connected layers are, for example, specific for determining the output string or characters/symbols 155 at a specific location in the registered country. For example, registrations that are shorter than the number of maximum characters/symbols 155 understood by the network 150 (somewhere between 7 and 10 depending on the country covered) are filled with spaces at the end. The additional output branch 225 is responsible for predicting the country from the set of countries for which a particular instance of the network has been trained. It should be noted that the convolutional layers, like the layers of block 200, serve as detectors for individual characters/symbols 155 of different sizes, regardless of where they appear in image 135. The fully connected layer processes the resulting heatmap, as does layer 230 or 245, to extract the correctly ordered sequence. The shared fully-connected layer, like layer 210, allows for efficient information exchange between different locations and/or national branches (branches 220 or 225, respectively).

The network is trained, for example, using data enhancement techniques, to ensure a continuously high recognition rate across different locations within the cropped area.

Avoiding the error-prone intermediate stage of character segmentation, as in the methods disclosed herein, greatly improves the accuracy of predictive reading and eliminates the need to mark character positions in training data. This makes it easy to obtain a larger amount of training data, thereby further improving the accuracy of the output.

Locating the license plate anywhere within the input image rather than filling it completely avoids problems due to license plate positioning errors (the immediately preceding step of the ANPR pipeline). It also makes it easy to handle two rows of plates whose input does not fit in a network that exactly matches the shape of a single row of plates.

The entire method for both reading and country identification allows the grammar rules to be implicitly embedded in the network and produce the most likely output combination. The country identification may take into account all aspects of visual appearance, such as font, word-breaking indicia, spacing of character sets, and any special symbols related to the issuing jurisdiction.

Fig. 3 shows a flow diagram of a method 300 for identifying a license plate of a vehicle according to an embodiment. The method 300 comprises a step 310 of reading in an image of the surroundings of an optical sensor originating from the optical sensor taking a picture of at least said vehicle with a detectable license plate. Further, the method 300 includes a step 320 of analyzing the image using a convolutional neural network having at least two separate symbol identification branches, each of the separate symbol identification branches configured to identify one of the number of symbols of the license plate and/or a country identification branch separate from the symbol identification branches configured to identify the country that has issued the license plate. Finally, the method 300 includes a step 330 of outputting the at least two identified symbols and/or the identified country for identification of the vehicle license plate.

Fig. 4 shows a flow diagram of a method 400 for training a convolutional neural network for use in a method for identifying a license plate of a vehicle according to an embodiment. The method 400 comprises a step 410 of reading in an image of the surroundings of an optical sensor, which image originates from the optical sensor taking a picture of at least the vehicle with a detectable license plate. Further, the method 400 includes a step 420 of providing a convolutional neural network having at least two separate symbol identification branches, each of the separate symbol identification branches configured to identify one of the number of symbols of the license plate and/or a country identification branch separate from the symbol identification branches configured to identify the country that has issued the license plate. Finally, method 400 includes a step 430 of training the convolutional neural network using the image read in.

If the exemplary embodiment comprises an "and/or" link between the first feature and the second feature, this will be understood as: this embodiment includes both the first feature and the second feature when in accordance with one embodiment, and includes only the first feature or only the second feature when in accordance with another embodiment.