阅读说明：本技术 基于轨旁设备的列车测速方法及装置 (Train speed measuring method and device based on trackside equipment ) 是由 张强 张宇旻 余祖俊 于 2021-10-29 设计创作，主要内容包括：本申请提供一种基于轨旁设备的列车测速方法及装置。所述方法包括：从通过轨旁设备的相机获取到的轨道区的当前二维图像中,确定列车所处的当前目标区域；将通过轨旁设备的激光雷达获取到的轨道区的当前点云图像,投影至当前二维图像所处的二维平面,获取当前点云图像中投影至当前目标区域的各当前目标点；根据各当前目标点确定列车的当前位置信息；根据当前位置信息、由各历史目标点确定的列车的历史位置信息以及目标时间间隔,确定列车的运行速度。本申请实施例提供的基于轨旁设备的列车测速方法能够避免毫米波雷达对于低速基于轨旁设备的列车测速精度不高,并且容易受到其他运动物体干扰的问题,提高检测到的列车速度的准确性。(The application provides a train speed measuring method and device based on trackside equipment. The method comprises the following steps: determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment; projecting a current point cloud image of the track area acquired by a laser radar of the trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to a current target area in the current point cloud image; determining the current position information of the train according to each current target point; and determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval. The train speed measuring method based on the trackside equipment can avoid the problems that the speed measuring precision of a low-speed train based on the trackside equipment is not high and the train is easily interfered by other moving objects due to the millimeter wave radar, and improves the accuracy of the detected train speed.)

1. A train speed measuring method based on trackside equipment is characterized by comprising the following steps:

determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment;

projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

determining the current position information of the train according to each current target point;

determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

2. The method for measuring the speed of the train based on the trackside equipment as claimed in claim 1, wherein the step of determining the current target area where the train is located from the current two-dimensional image of the track area acquired by the camera of the trackside equipment comprises the steps of:

inputting the current two-dimensional image into a trained train identification model for train detection, and acquiring a train image in the current two-dimensional image;

and determining the current target area of the train in the current two-dimensional image according to the position of the train image in the two-dimensional image.

3. The method for measuring the speed of the train based on the trackside equipment as claimed in claim 1, wherein the step of projecting the current point cloud image of the track area acquired by the laser radar of the trackside equipment to the two-dimensional plane where the current two-dimensional image is located comprises the steps of:

carrying out combined calibration on the camera and the laser radar to obtain internal parameters and external parameters of the camera;

and performing coordinate conversion on the current point cloud image according to the internal parameters and the external parameters, and projecting the current point cloud image to a two-dimensional plane where the current two-dimensional image is located so as to enable the current point cloud image to be overlapped with the current two-dimensional image.

4. The method for measuring the speed of the train based on the trackside equipment as claimed in claim 1, wherein the determining the current position information of the train according to each current target point comprises:

superposing each current target point to a preset map of the track area;

extracting each appointed characteristic point positioned in the track area of the preset map from each current target point;

determining the current position information of the train according to each designated feature point;

and establishing the preset map based on a laser radar coordinate system of the laser radar.

5. The method for measuring the speed of the train based on the trackside equipment as claimed in any one of claims 1 to 4, wherein the time stamp of the current two-dimensional image is synchronized with the time stamp of the current point cloud image, and the time stamp of the historical two-dimensional image is synchronized with the time stamp of the historical point cloud image.

6. The method for measuring the speed of the train based on the trackside equipment as claimed in claim 5, wherein the time stamp of the current two-dimensional image is synchronized with the time stamp of the current point cloud image, and the time stamp of the historical two-dimensional image is synchronized with the time stamp of the historical point cloud image, and the method comprises the following steps:

the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image is not more than a first preset value;

the interval between the time stamp of the historical two-dimensional image and the time stamp of the historical point cloud image is not more than a first preset value;

the first preset value is determined according to the frame rate of the camera and the frame rate of the laser radar.

7. The method for measuring the speed of the train based on the trackside equipment as claimed in any one of claims 1 to 5, wherein N frames of point cloud images are arranged between the current point cloud image and the historical point cloud image, and N is a positive integer.

8. The utility model provides a train speed sensor based on trackside equipment which characterized in that includes:

the area determining module is used for determining a current target area where the train is located from a current two-dimensional image of the track area acquired by a camera of the trackside equipment;

the image projection module is used for projecting the current point cloud image of the track area, which is acquired by a laser radar of the trackside equipment, to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

the position determining module is used for determining the current position information of the train according to each current target point;

the train speed measuring module is used for determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

9. An electronic device comprising a processor and a memory storing a computer program, wherein the processor when executing the computer program implements the steps of the trackside device-based train speed measurement method of any of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the method for trackside device-based train speed measurement according to any one of claims 1 to 7.

Technical Field

The application relates to the technical field of image processing, in particular to a train speed measuring method and device based on trackside equipment.

Background

At present, in the process of running of a train, in order to ensure the running safety of the train, speed measurement needs to be carried out on the train through trackside equipment.

In the related art, the trackside equipment usually employs millimeter wave radar or train-ground communication to acquire the train speed. The millimeter wave radar is installed beside a track of a train track and used for emitting and receiving electromagnetic waves in the direction of coming vehicles facing to one side of the track and then measuring the speed of the coming vehicles through the Doppler effect.

However, since the speed acquired by the millimeter wave radar is an integral multiple of a certain lowest speed when the train runs at a low speed, that is, the step amount of the lowest speed when the train runs at a low speed, the continuously changing speed cannot be output. When the real speed of the train is between two adjacent step quantities, the output speed of the millimeter wave radar is lower or higher. In addition, when other moving objects besides the train exist in the scene, the speed measurement of the millimeter wave radar is also easily interfered, so that the acquired train speed is inaccurate.

Disclosure of Invention

The embodiment of the application provides a train speed measuring method and device based on trackside equipment, which can avoid the problems that the speed measuring precision of a millimeter wave radar for a low-speed train based on trackside equipment is not high and the train is easily interfered by other moving objects, and improve the accuracy of the detected train speed.

In a first aspect, an embodiment of the present application provides a train speed measurement method based on trackside equipment, including:

determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment;

projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

determining the current position information of the train according to each current target point;

determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

In one embodiment, determining a current target area where the train is located from a current two-dimensional image of the track area acquired by a camera of the trackside device includes:

inputting the current two-dimensional image into a trained train identification model for train detection, and acquiring a train image in the current two-dimensional image;

and determining the current target area of the train in the current two-dimensional image according to the position of the train image in the two-dimensional image.

In one embodiment, the projecting the current point cloud image of the track area acquired by the lidar of the trackside device to the two-dimensional plane where the current two-dimensional image is located includes:

carrying out combined calibration on the camera and the laser radar to obtain internal parameters and external parameters of the camera;

and performing coordinate conversion on the current point cloud image according to the internal parameters and the external parameters, and projecting the current point cloud image to a two-dimensional plane where the current two-dimensional image is located so as to enable the current point cloud image to be overlapped with the current two-dimensional image.

In an embodiment, the determining the current location information of the train according to each of the current target points includes:

superposing each current target point to a preset map of the track area;

extracting each appointed characteristic point positioned in the track area of the preset map from each current target point;

determining the current position information of the train according to each designated feature point;

and establishing the preset map based on a laser radar coordinate system of the laser radar.

In one embodiment, the timestamp of the current two-dimensional image is synchronized with the timestamp of the current point cloud image, and the timestamp of the historical two-dimensional image is synchronized with the timestamp of the historical point cloud image.

In one embodiment, the time stamp of the current two-dimensional image synchronized with the time stamp of the current point cloud image and the time stamp of the historical two-dimensional image synchronized with the time stamp of the historical point cloud image comprise:

the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image is not more than a first preset value;

the interval between the time stamp of the historical two-dimensional image and the time stamp of the historical point cloud image is not more than a first preset value;

the first preset value is determined according to the frame rate of a camera used for obtaining the current two-dimensional image and the frame rate of a laser radar used for obtaining the current point cloud image.

In one embodiment, N frames of point cloud images are arranged between the current point cloud image and the historical point cloud image, and N is a positive integer.

In a second aspect, an embodiment of the present application provides a train speed measurement device based on trackside equipment, including:

the area determining module is used for determining a current target area where the train is located from a current two-dimensional image of the track area acquired by a camera of the trackside equipment;

the image projection module is used for projecting the current point cloud image of the track area, which is acquired by a laser radar of the trackside equipment, to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

the position determining module is used for determining the current position information of the train according to each current target point;

the train speed measuring module is used for determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory storing a computer program, where the processor implements the steps of the method for measuring speed of a train based on a trackside device according to the first aspect when executing the program.

In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the method for measuring speed of a train based on a trackside device in the first aspect are implemented.

According to the train speed measuring method and device based on the trackside equipment, the point cloud image of the track area is projected to the two-dimensional image of the track area, after the point cloud image projected to the target area of the train in the two-dimensional image is obtained from the point cloud image, the position of the train in the point cloud image is determined through the target points, and the running speed of the train is obtained according to the position of the train in the two frames of point cloud images and the time interval between the two frames of point cloud images, so that when the train runs at a low speed, the position of the train in the point cloud images at different moments can be changed, and the speed of the train can be effectively detected through the position change of the train in the different point cloud images. Meanwhile, only processing each target point of the target area where the train is located, so that the problem that when other moving objects exist in the scene except the train, the train is interfered by the other moving objects when the train is tested, and the accuracy of the detected train speed is improved.

Drawings

In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic application environment diagram of a train speed measurement method based on trackside equipment provided by an embodiment of the present application;

FIG. 2 is a schematic flow chart of a train speed measurement method based on trackside equipment provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a position deviation between a two-dimensional image and a point cloud image provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a checkerboard provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a point cloud coverage provided by an embodiment of the present application;

fig. 6 is a schematic structural diagram of a train speed measuring device based on trackside equipment provided by an embodiment of the application;

fig. 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

Detailed Description

To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

For a better understanding of the solution, the technical terms to which the embodiments of the present invention relate are explained.

Laser radar (LiDAR) is a short for laser Detection and Ranging system, and is a product of combining laser technology and radar technology. The laser radar adopts a laser as a radar of a radiation source, and generally comprises a transmitter, an antenna, a receiver, a tracking frame, information processing and the like. The transmitter is a laser of various forms; the antenna is an optical telescope; the receiver adopts various forms of light spot detectors; the laser radar adopts two working modes of pulse or continuous wave, and the detection method is divided into direct detection and heterodyne detection. A LiDAR system includes a single beam narrowband laser and a receiving system. The laser generates and emits a beam of light pulses which impinge on the object and are reflected back and finally received by the receiver. The receiver accurately measures the travel time of the light pulse from emission to reflection. Because the light pulses travel at the speed of light, the receiver will always receive the previous reflected pulse before the next pulse is sent out. Given that the speed of light is known, the travel time can be converted into a measure of distance. And by combining the height of the laser, the laser scanning angle, the position of the laser obtained from the GPS and the laser emission direction obtained from the INS, the coordinates X, Y and Z of each ground light spot can be accurately calculated. The frequency of laser beam emission can range from a few pulses per second to tens of thousands of pulses per second. For example, in a system with a frequency of ten thousand pulses per second, the receiver would record sixty thousand points in one minute. In general, the ground spot spacing of LiDAR systems varies from 2-4 m. Lidar is a radar system that operates in the infrared to ultraviolet spectral range and has a principle and construction very similar to a laser range finder. The laser radar is used for accurately measuring the position (distance and angle), the motion state (speed, vibration and attitude) and the shape of a target, and detecting, identifying, distinguishing and tracking the target.

The Point Cloud is a massive Point set which expresses target space distribution and target surface characteristics under the same space reference system, and after the space coordinates of each sampling Point on the surface of the object are obtained, the Point set is obtained and is called as the Point Cloud. The method mainly acquires point cloud data by acquiring data through a three-dimensional laser scanner, and the LiDAR (laser radar) acquires the point cloud data, and simultaneously processes and applies the point cloud data.

The embodiments of the present application will be described in detail below with reference to the accompanying drawings. The method for measuring the speed of the train based on the trackside equipment provided by the embodiment of the application is applied to an application environment comprising a train track 110 and trackside equipment as shown in fig. 1. Wherein, trackside equipment includes: camera 120, lidar 130, and computer device 140, camera 120, lidar 130, and computer device 140 are disposed beside train track 110. The camera may specifically be an electronic device having a camera shooting or recording function, such as a digital camera or a video camera. The computer device 140 may be a terminal device or a server, where the terminal device may be a desktop terminal or a mobile terminal, and the mobile terminal may be one of any mobile terminals such as a mobile phone, a tablet computer, and a notebook computer; the server may be an independent server or a server group composed of a plurality of servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like.

The camera of the trackside equipment is used for acquiring a current two-dimensional image of a track area in the coming direction of one side of the train track, and the laser radar of the trackside equipment is used for acquiring a current point cloud image of the track area. After the current two-dimensional image and the current point cloud image are obtained, a processing device of the trackside device determines a current target area where the train is located from the current image, then the current point cloud image is projected to a two-dimensional plane where the current two-dimensional image is located, and point cloud data projected to the current target area in the current point cloud image, namely current target points, are obtained. After each current target point is obtained, the current position information of the train is determined according to the current target point, and then the running speed of the train is determined based on the current position information, the historical position information of the train determined by the historical target point and the target time interval.

The historical target point is obtained as the current target point, the historical two-dimensional image of the track area is acquired by a camera of the trackside equipment, the historical point cloud image of the track area is acquired by a laser radar of the trackside equipment, the historical target area where the train is located is determined from the historical two-dimensional image through a processing device of the trackside equipment, then the historical point cloud image is projected to the two-dimensional plane where the historical two-dimensional image is located, and the point cloud data projected to the historical target area in the historical point cloud image, namely all the historical target points, are obtained.

The historical two-dimensional image is the first N frames of two-dimensional images of the current two-dimensional image, the historical point cloud image is the first N frames of point cloud images of the current point cloud image, and N is a positive integer.

The target time interval is the time interval between the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

The method comprises the steps of projecting a point cloud image of a track area to a two-dimensional image of the track area, obtaining target points projected to a target area where the two-dimensional image of a train is located from the point cloud image, determining the position of the train in the point cloud image through the target points, and obtaining the running speed of the train according to the position of the train in two frames of the point cloud image and the time interval between the two frames of the point cloud image, so that the position of the train in the point cloud image at different moments can be changed when the train runs at a low speed, and the speed of the train can be effectively detected through the position change of the train in different point cloud images. Meanwhile, only processing each target point of the target area where the train is located, so that the problem that when other moving objects exist in the scene except the train, the train is interfered by the other moving objects when the train is tested, and the accuracy of the detected train speed is improved.

The train speed measuring method based on the trackside equipment provided by the embodiment of the application is described and explained in detail through several specific embodiments.

Referring to fig. 2, a schematic flow chart of a train speed measurement method based on trackside equipment according to an embodiment of the present invention is shown, where the method is applied to a computer device for detecting a speed of a train. The computer device may specifically be the computer device 140 in fig. 1.

As shown in fig. 2, the train speed measurement method based on the trackside equipment provided in this embodiment includes:

step 101, determining a current target area where a train is located from a current two-dimensional image of a track area acquired by a camera of trackside equipment;

102, projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

103, determining the current position information of the train according to each current target point;

104, determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

The method comprises the steps of projecting a point cloud image of a track area to a two-dimensional image of the track area, obtaining target points projected to a target area where the two-dimensional image of a train is located from the point cloud image, determining the position of the train in the point cloud image through the target points, and obtaining the running speed of the train according to the position of the train in two frames of point cloud images and the time interval between the two frames of point cloud images. Meanwhile, the limit intrusion detection of the track area can be realized by utilizing a laser radar and a camera.

In step 101, the current two-dimensional image is a two-dimensional image acquired by a camera disposed beside the track. The camera shoots the coming direction of the track to obtain the current two-dimensional image of the track area. The frame rate of the camera can be set according to actual conditions, and can be set to be about 25Hz as an example.

Because the two-dimensional image acquired by the camera usually has not only trains but also buildings, pedestrians and other areas which do not need to be processed, the trackside equipment acquires the current two-dimensional image by the camera, and then preprocesses the current two-dimensional image to determine the current target area of the train in the current two-dimensional image. Specifically, the position and the size of the train can be detected from the current two-dimensional image through the trained train recognition model, and then the current target area of the train in the current two-dimensional image is determined.

In order to accurately determine the current target area where the train is located, in an embodiment, determining the current target area where the train is located from the current two-dimensional image of the track area acquired by the camera of the trackside device includes:

inputting the current two-dimensional image into a trained train identification model for train detection, and acquiring a train image in the current two-dimensional image;

and determining the current target area of the train in the current two-dimensional image according to the position of the train image in the two-dimensional image.

In an embodiment, the trained train recognition model may be a model for recognizing a vehicle, which is subjected to parameter training by using a transfer learning method, or a plurality of conventional image recognition models, such as SSD, YOLOv3, YOLOv4, YOLOv5, and the like, are trained in advance through a large number of positive and negative samples, so that an image recognition model with the highest detection accuracy and capable of reaching a preset accuracy, such as 95%, is selected as the trained train recognition model. In order to ensure the credibility of the training samples, the positive and negative samples can adopt massive drive test data, namely original data which is obtained based on a real scene and is not subjected to image processing, so that the credibility of the data is greatly improved. After acquiring the massive original data, the original data can be marked in advance according to actual conditions, and the massive original data is marked as a positive sample or a negative sample.

In order to further improve the accuracy of the model detection, in an embodiment, the train identification model may be further appropriately adjusted until the train identification model generates more accurate detections, and then the two-dimensional image acquired by the camera is filtered by using relevant regularity restrictions according to the adjusted content. If a train is detected, the train is always on the track, otherwise the train is not detected, and the train can be screened out. Thereby further improving the accuracy of model detection.

In an embodiment, the current two-dimensional image is input into a trained wheel recognition model for train detection, and a train image in the two-dimensional image can be obtained. After the train image is obtained, the area formed by the coordinates of all pixels of the train image can be used as the position of the train in the two-dimensional image. If the two-dimensional coordinates (x1, y1), (x2, y2), (x3, y3) … … (xn, ym) of each pixel point in the train image are obtained, the pixel points are used as the positions of the train in the two-dimensional image. Or, after the train image is obtained, the vertex coordinates of the circumscribed two-dimensional frame of the train image are used as the position of the train image in the two-dimensional image. If the vertex coordinates of the two-dimensional frame are (x1, y1), (x2, y2), (x3, y3) and (x4, y4), the position of the two-dimensional frame enclosed by the four vertex coordinates is the position of the train image in the two-dimensional image.

After the position of the train image in the two-dimensional image is obtained, the two-dimensional image is cut according to the position, the display area or the external two-dimensional frame of the display area is extracted from the two-dimensional image to serve as the current target area, so that the target area where the train is located can be screened out, the image in the target area is only processed subsequently, and the influence of other moving objects on the subsequent detection result is avoided.

In step 102, after a current target area where the train is located is determined, a current point cloud image of the track area acquired by the laser radar is projected to a two-dimensional plane where the current two-dimensional image is located. And synchronizing the time stamp of the current two-dimensional image with the time stamp of the current point cloud image. The timestamp synchronization may refer to that the current point cloud image and the current two-dimensional image are images at the same time, that is, the timestamp of the current two-dimensional image is the same as the timestamp of the current point cloud image. Thus, when the current point cloud image is projected to the two-dimensional plane where the current two-dimensional image is located, the projection deviation is minimum.

However, if the current point cloud image and the current two-dimensional image are required to be images at the same time, the camera needs to support line control triggering shooting, and at the moment, the image frame of the camera and the laser radar point cloud frame can be completely aligned in time, so that the current point cloud image and the current two-dimensional image at the same time can be obtained. If the camera does not support drive-by-wire triggered shooting, the point cloud output by the laser radar and the camera image cannot be completely the same in time due to the fact that frame rates of the laser radar and the camera are different, and at the moment, the projection deviation is large, and the accuracy of subsequent train speed measurement based on trackside equipment is affected. To this end, in an embodiment, the synchronizing the time stamp of the current two-dimensional image and the time stamp of the current point cloud image includes:

the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image is not larger than a first preset value. The first preset value is determined according to the frame rate of the camera and the frame rate of the laser radar.

Assuming that the frame rate of the camera is 25Hz and the frame rate of the lidar is 10Hz, the first preset value is theoretically half the period of the image frame of the camera, i.e. 20ms, of the maximum time difference between the two-dimensional image and the current point cloud image. For example, if the maximum speed of the train is 80 km/h, the maximum travel distance of the train is 0.44 m within 20 ms. And the camera and the radar face the coming train direction, as shown in fig. 3, so that when the current point cloud image is projected onto the current two-dimensional image, the position deviation can be ignored. I.e., from the camera or radar perspective, the horizontal displacement of the train 0.44 meters forward on the image is negligible. Therefore, when the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image is not greater than the first preset value, the time stamp of the current two-dimensional image and the time stamp of the current point cloud image can be considered to be synchronous. At the moment, the current point cloud image of which the time interval with the timestamp of the current two-dimensional image is not greater than the first preset value can be obtained and mapped to the two-dimensional plane where the current two-dimensional image is located, so that the condition that the projection deviation is overlarge is avoided, and the influence on the accuracy of subsequent train speed measurement based on trackside equipment is avoided. Preferably, the current point cloud image may be the point cloud image with the closest timestamp to the current two-dimensional image.

In an embodiment, the current point cloud image is projected to a two-dimensional plane where the current two-dimensional image is located, and the camera and the laser radar may be jointly calibrated to obtain internal parameters and external parameters of the camera;

and performing coordinate conversion on the current point cloud image according to the internal parameters and the external parameters, and projecting the current point cloud image to a two-dimensional plane where the current two-dimensional image is located so as to enable the current point cloud image to be overlapped with the current two-dimensional image.

In one embodiment, before projection, joint calibration is performed on the laser radar and the camera to obtain calibration parameters of the camera, such as external parameters and internal parameters. Then, during projection, the current point cloud image and the current two-dimensional image are registered through the calibration parameters to obtain the spatial position relationship of the two images, and then the three-dimensional current point cloud image is converted into two-dimensional scatter points in a two-dimensional coordinate system where the current two-dimensional image is located based on the spatial position relationship, so that the current point cloud image and the region of the current two-dimensional image representing the same object are completely overlapped.

In one embodiment, the current point cloud image is first processed by external parametersFrom the radar coordinate system (x)_w,y_w,z_w) Conversion into camera coordinate system (x)_c,y_c,z_c)：

Where R is a rotation matrix, it may be represented by rotation in three directions (Pitch, Roll, Yaw) in a rectangular coordinate system. t is the translation vector, i.e. the distance in three directions in a rectangular coordinate system (t)_x,t_y,t_z) The rotation plus translation total six parameters, constituting the extrinsic parameters.

Then, a point (x) in the camera coordinate system is imaged by the internal parameters_c,y_c,z_c) Into pixel coordinates (u, v):

wherein f is_xAnd f_yIs the focal length of the camera in both the horizontal and vertical directions, u₀And v₀Is the pixel coordinate of the camera center, s is the axis tilt coefficient, and a total of five parameters constitute the internal parameters.

In one embodiment, the internal parameters are only relevant to the camera. The internal parameters can be determined by shooting a plurality of checkerboard pictures with different postures by a camera and then inputting the pictures into the existing standard detection algorithm. Wherein the checkerboard picture may be as shown in fig. 4.

The extrinsic parameters relate to the relative position of the lidar and the camera. Usually, the point cloud image obtained by the laser radar is projected onto the two-dimensional image obtained by the camera according to the measured internal parameters and a set of external parameter estimation values determined by experience, the external parameters are adjusted to make the point cloud image of the same object completely coincide with the outline of the two-dimensional image, and then the final external parameters are obtained.

In an embodiment, after the current point cloud image is projected onto a two-dimensional plane where the current two-dimensional image is located, point cloud data falling on a current target area, that is, each current target point, can be extracted. And at the moment, each current target point is point cloud data corresponding to the train.

Since the current target area where the train is located, which is determined from the current two-dimensional image, may be a little larger, all of the current target points may not be target points representing the train, that is, the current target points may include point cloud data on a background behind the train. In addition, it is also possible that the coverage of each current target point may contain point cloud data falling on the obstruction, as shown in fig. 5, due to other obstructions between the train and the lidar, such as utility poles or pedestrians. Therefore, the current position information of the train is determined by directly utilizing each current target point, which causes the current position information to be inaccurate, and further influences the accuracy of the subsequently detected train speed. To this end, in an embodiment, determining the current location information of the train according to each of the current target points includes:

superposing each current target point to a preset map of the track area;

extracting each appointed characteristic point positioned in the track area of the preset map from each current target point;

determining the current position information of the train according to each designated feature point;

and establishing the preset map based on a laser radar coordinate system of the laser radar.

In one embodiment, a preset map of a track area of a laser radar coordinate system based on a laser radar is pre-established, wherein the preset map can be a three-dimensional map or a top view of a train track, then, current target points are superposed on the preset map, and the preset map is segmented and clustered into three targets according to the distribution of point cloud. Since only the train is located on the track, each specified feature point located in the track area of the preset map representing the track can be extracted from the current target point. And at the moment, each extracted specified characteristic point is point cloud data representing the train.

And after each appointed characteristic point is extracted, determining the current position information of the train according to each appointed characteristic point.

For example, an average coordinate point may be obtained from coordinate points of each specified feature point in the laser radar coordinate system, and then the average coordinate point may be used as the current position information of the train.

By superposing the current target points on the preset map, the specified characteristic points in the track area are extracted from the current target points, so that point cloud data only containing the train can be extracted, the accuracy of the obtained current position information is improved, and the accuracy of the subsequently detected train speed is improved.

In order to make the acquired current position information more accurate, in an embodiment, for the acquisition of the current position information, the center of mass of the train can be acquired according to the coordinate information of each specified characteristic point in a laser radar coordinate system;

and determining the coordinate information of the centroid in a laser radar coordinate system as the current position information of the train.

In step 104, after the current position information is obtained, the running distance d of the train during the interval between the current point cloud image and the historical point cloud image with the historical target points can be obtained according to the current position information and the historical position information of the train determined by the historical target points. And calculating a target time interval t according to the time stamps of the current point cloud image and the historical point cloud image, so that the current running speed v of the train can be calculated to be d/t.

And each historical target point is projected to each target point of the historical target area of the historical two-dimensional image of the track area in the historical point cloud image of the track area. The interval between the time stamp of the historical two-dimensional image and the time stamp of the historical point cloud image is the same as the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image, and the interval is not more than a first preset value.

The method for acquiring each historical target point is the same as the method for acquiring each current target point, and only the current two-dimensional image is replaced by the historical two-dimensional image, and the current point cloud image is replaced by the historical point cloud image, which is not repeated herein.

The method for acquiring the historical position information is the same as the method for acquiring the current position information, and only the current target points are replaced by the historical target points, which is not repeated herein.

Similarly, the time stamp of the historical two-dimensional image is synchronous with the time stamp of the historical point cloud image. The time stamp synchronization may also mean that the historical point cloud image and the historical two-dimensional image are images at the same time, that is, the time stamp of the historical two-dimensional image is the same as the time stamp of the historical point cloud image. Thus, when the historical point cloud image is projected to the two-dimensional plane where the historical two-dimensional image is located, the projection deviation is minimum. Or the interval between the time stamp of the historical two-dimensional image and the time stamp of the historical point cloud image is not greater than a first preset value.

It is considered that if the number of frames spaced between the current two-dimensional image and the historical two-dimensional image is too small, if the current two-dimensional image and the historical two-dimensional image are adjacent two-dimensional images, the running distance d of the train obtained at this time may be too small, for example, less than 5cm, if the train speed is very slow. At this time, the accuracy of measurement of an extremely low vehicle speed may be reduced due to an insignificant running distance of the train. For this purpose, in one embodiment, N frames of point cloud images are spaced between the current point cloud image and the historical point cloud image, where N is a positive integer.

For example, 1 frame, 5 frames or 10 frames can be spaced between the current point cloud image and the historical point cloud image, so that the frame number interval also exists between the current two-dimensional image and the historical two-dimensional image, the driving distance of the train is calculated by spacing the images of multiple frames at the moment, the driving distance d of the train can be more remarkable, and the interval time t can be synchronously increased, so that the accurate measurement of the extremely low vehicle speed is realized.

By utilizing the frame skipping measurement of the laser radar point cloud, the obtained characteristic parameters of the train are more obvious, so that the train can still accurately obtain the running distance and the corresponding running time even if the train runs at an extremely low speed, and the accuracy of the detected train speed is further improved.

The train speed measuring device based on the trackside equipment provided by the embodiment of the application is described below, and the train speed measuring device based on the trackside equipment described below and the train speed measuring method based on the trackside equipment described above can be referred to correspondingly.

In an embodiment, as shown in fig. 6, there is provided a train speed measuring device based on a trackside device, including:

the area determining module is used for determining a current target area where the train is located from a current two-dimensional image of the track area acquired by a camera of the trackside equipment;

the image projection module is used for projecting the current point cloud image of the track area, which is acquired by a laser radar of the trackside equipment, to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

the position determining module is used for determining the current position information of the train according to each current target point;

the train speed measuring module is used for determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

In an embodiment, the area determination module 210 is specifically configured to: inputting the current two-dimensional image into a trained train identification model for train detection, and acquiring a train image in the current two-dimensional image;

and determining the current target area of the train in the current two-dimensional image according to the position of the train image in the two-dimensional image.

In one embodiment, the image projection module 220 is specifically configured to:

carrying out combined calibration on the camera and the laser radar to obtain internal parameters and external parameters of the camera;

and performing coordinate conversion on the current point cloud image according to the internal parameters and the external parameters, and projecting the current point cloud image to a two-dimensional plane where the current two-dimensional image is located so as to enable the current point cloud image to be overlapped with the current two-dimensional image.

In an embodiment, the position determining module 230 is specifically configured to:

superposing each current target point to a preset map of the track area;

extracting each appointed characteristic point positioned in the track area of the preset map from each current target point;

determining the current position information of the train according to each designated feature point;

and establishing the preset map based on a laser radar coordinate system of the laser radar.

In an embodiment, the position determining module 230 is specifically configured to:

acquiring the mass center of the train according to the coordinate information of each specified characteristic point in a laser radar coordinate system;

and determining the coordinate information of the centroid in a laser radar coordinate system as the current position information of the train.

In one embodiment, the timestamp of the current two-dimensional image is synchronized with the timestamp of the current point cloud image, and the timestamp of the historical two-dimensional image is synchronized with the timestamp of the historical point cloud image.

In one embodiment, the time stamp of the current two-dimensional image synchronized with the time stamp of the current point cloud image and the time stamp of the historical two-dimensional image synchronized with the time stamp of the historical point cloud image comprise:

the interval between the time stamp of the current two-dimensional image and the time stamp of the current point cloud image is not more than a first preset value;

the interval between the time stamp of the historical two-dimensional image and the time stamp of the historical point cloud image is not more than a first preset value;

the first preset value is determined according to the frame rate of the camera and the frame rate of the laser radar.

In one embodiment, N frames of point cloud images are arranged between the current point cloud image and the historical point cloud image, and N is a positive integer.

The method comprises the steps of projecting a point cloud image of a track area to a two-dimensional image of the track area, obtaining target points projected to a target area where the two-dimensional image of a train is located from the point cloud image, determining the position of the train in the point cloud image through the target points, and obtaining the running speed of the train according to the position of the train in two frames of point cloud images and the time interval between the two frames of point cloud images. Meanwhile, the limit intrusion detection of the track area can be realized by utilizing a laser radar and a camera.

Fig. 7 illustrates a physical structure diagram of an electronic device, and as shown in fig. 7, the electronic device may include: a processor (processor)810, a Communication Interface 820, a memory 830 and a Communication bus 840, wherein the processor 810, the Communication Interface 820 and the memory 830 communicate with each other via the Communication bus 840. The processor 810 may invoke computer programs in the memory 830 to perform the steps of the trackside device-based train speed measurement method, including, for example:

determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment;

projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

determining the current position information of the train according to each current target point;

determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present application further provides a computer program product, where the computer program product includes a computer program, where the computer program is stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, a computer is capable of executing the steps of the trackside device-based train speed measurement method provided in the foregoing embodiments, for example, the steps include:

determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment;

projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

determining the current position information of the train according to each current target point;

determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

On the other hand, embodiments of the present application further provide a processor-readable storage medium, where the processor-readable storage medium stores a computer program, where the computer program is configured to cause a processor to perform the steps of the method provided in each of the above embodiments, for example, including:

determining a current target area where a train is located from a current two-dimensional image of a track area acquired through a camera of the trackside equipment;

projecting a current point cloud image of the track area acquired by a laser radar of trackside equipment to a two-dimensional plane where the current two-dimensional image is located, and acquiring current target points projected to the current target area in the current point cloud image;

determining the current position information of the train according to each current target point;

determining the running speed of the train according to the current position information, the historical position information of the train determined by each historical target point and the target time interval;

each historical target point is projected to each target point of a historical target area of a historical two-dimensional image of the track area in the historical point cloud image of the track area;

the historical point cloud image is collected through the camera, and the historical two-dimensional image passes through the laser radar;

the historical target area is an area where the train is located in the historical two-dimensional image;

the target time interval is determined according to the time stamp of the current point cloud image and the time stamp of the historical point cloud image.

The processor-readable storage medium can be any available medium or data storage device that can be accessed by a processor, including, but not limited to, magnetic memory (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical memory (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), Solid State Disks (SSDs)), etc.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.