阅读说明：本技术 一种信息处理方法、装置、电子设备及智能交通系统 (Information processing method and device, electronic equipment and intelligent traffic system ) 是由 罗义平 于 2018-06-08 设计创作，主要内容包括：本申请提供一种信息处理方法、装置、电子设备及智能交通系统,该方法包括：基于视频检测方式识别视频图像中的人行信号灯的状态；基于深度学习算法对视频图像进行目标检测,得到目标检测数据；所述目标包括行人或非机动车；根据目标检测数据以及人行信号灯的状态确定视频图像中是否存在目标闯红灯行为；若存在,则将存在闯红灯行为的目标的人脸图像发送至后端服务器,由所述后端服务器生成针对该目标的闯红灯取证信息。该方法可以提高闯红灯行为检测的检出率和有效率,扩展方案的应用场景。(the application provides an information processing method, an information processing device, electronic equipment and an intelligent traffic system, wherein the method comprises the following steps: identifying the state of a pedestrian signal lamp in a video image based on a video detection mode; performing target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle; determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp; and if the target exists, sending the face image of the target with the red light running behavior to a back-end server, and generating red light running evidence obtaining information aiming at the target by the back-end server. The method can improve the detection rate and the effective rate of the red light running behavior detection, and expand the application scene of the scheme.)

1. an information processing method is applied to front-end video acquisition equipment in an intelligent transportation system, and is characterized by comprising the following steps:

Identifying the state of a pedestrian signal lamp in a video image based on a video detection mode;

performing target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

Determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp;

And if the target exists, sending the face image of the target with the red light running behavior to a back-end server, and generating red light running evidence obtaining information aiming at the target by the back-end server.

2. The method according to claim 1, wherein before the identifying the state of the pedestrian signal lamp in the video image based on the video detection mode and the target detection of the video image based on the deep learning algorithm, the method further comprises:

Preprocessing a video image;

the state of the pedestrian signal lamp in the video image is identified based on the video detection mode, and the method comprises the following steps:

identifying the state of a pedestrian signal lamp in the preprocessed video image based on a video detection mode;

the target detection of the video image based on the deep learning algorithm comprises the following steps:

and carrying out target detection on the preprocessed video image based on a deep learning algorithm.

3. The method of claim 2, wherein the pre-processing the video image comprises one or more of:

Noise reduction, filtering, format conversion, and down-sampling.

4. the method of claim 1, wherein determining whether a target red light running behavior exists in the video image according to the target detection data and the status of the pedestrian signal light comprises:

For any target in the video image, determining the action track information of the target according to the target detection data of the video image comprising the target;

and determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp.

5. the method as claimed in claim 4, wherein the determining whether the object has the red light running behavior according to the action track information of the object and the state of the pedestrian signal lamp comprises:

and if the pedestrian signal lamp is in a red light state, the target enters a pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing is greater than a preset distance threshold, and the target is determined to have a red light running behavior.

6. the method as claimed in claim 5, wherein the determining whether the object has the red light running behavior according to the action track information of the object and the state of the pedestrian signal light further comprises:

And if the pedestrian signal lamp is in a red light state, the target enters a pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing is smaller than or equal to the preset distance threshold, determining that the target has a red light running trend, and performing early warning prompt.

7. The method according to claim 1, wherein the sending the face image of the target with the red light running behavior to a back-end server comprises:

for any target with red light running behavior, respectively intercepting small images comprising the target from a preset number of video images comprising the target, and splicing the intercepted small images to obtain a spliced image;

performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

And performing image quality grading on each face image based on a deep learning algorithm, and sending the face image with the highest grade to a back-end server.

8. The method of claim 7, wherein the image quality scoring of each face image based on the deep learning algorithm comprises:

Image quality scoring of the face image is performed according to one or more of the following parameters:

The human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

9. an information processing device is applied to front-end video acquisition equipment in an intelligent transportation system, and is characterized by comprising:

the signal lamp state detection unit is used for identifying the state of a pedestrian signal lamp in the video image based on a video detection mode;

the target detection unit is used for carrying out target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

the behavior judgment unit is used for determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp;

And the communication unit is used for sending the face image of the target with the red light running behavior to a back-end server if the target with the red light running behavior exists, and generating red light running evidence obtaining information aiming at the target by the back-end server.

10. the apparatus of claim 9, further comprising:

the preprocessing unit is used for preprocessing the video image;

the signal lamp state detection unit identifies the state of a pedestrian signal lamp in the preprocessed video image based on a video detection mode;

And the target detection unit is used for carrying out target detection on the preprocessed video image based on a deep learning algorithm.

11. the apparatus of claim 10,

The preprocessing unit is specifically configured to perform one or more of the following processing on the video image:

noise reduction, filtering, format conversion, and down-sampling.

12. The apparatus of claim 9,

the target detection unit is also used for determining the action track information of any target in the video image according to the target detection data of the video image comprising the target;

The behavior determination unit is specifically configured to determine whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp.

13. the apparatus of claim 12,

The behavior determination unit is specifically configured to determine that a red light running behavior exists in the target if the target enters the pedestrian crossing area and the distance between the target and the initial position of the pedestrian crossing is greater than a preset distance threshold value when the status of the pedestrian signal lamp is a red light status.

14. The apparatus of claim 13,

The behavior judging unit is used for determining that the target has a red light running trend if the target enters a pedestrian crossing area and the distance between the target and the initial position of the pedestrian crossing is less than or equal to the preset distance threshold value when the state of the pedestrian signal lamp is a red light state;

the device further comprises:

And the early warning prompting unit is used for giving early warning prompt when the target is determined to have the red light running trend.

15. The apparatus of claim 8, further comprising:

The human face detection unit is used for respectively intercepting small images comprising the target from a preset number of video images comprising the target for any target with the red light running behavior, and splicing the intercepted small images to obtain a spliced image; performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

The human face image scoring unit is used for scoring the image quality of each human face image based on a deep learning algorithm;

the communication unit is specifically configured to send the face image with the highest score to a back-end server.

16. the apparatus of claim 15,

The face image scoring unit is specifically configured to score the face image according to one or more of the following parameters:

the human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

17. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

A memory for storing a computer program;

a processor for implementing the method of any one of claims 1 to 8 when executing a program stored in a memory.

18. an intelligent traffic system is characterized by comprising front-end video acquisition equipment and a back-end server; wherein:

The video acquisition equipment is used for identifying the state of a pedestrian signal lamp in a video image based on a video detection mode;

the video acquisition equipment is also used for carrying out target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

the video acquisition equipment is also used for determining whether a target red light running behavior exists in a video image according to the target detection data and the state of the pedestrian signal lamp; if the target exists, sending the face image of the target with the red light running behavior to the back-end server;

and the back-end server is used for generating red light running evidence obtaining information aiming at the target when receiving the human face image of the target with the red light running behavior sent by the video acquisition equipment.

Technical Field

The present application relates to intelligent transportation technologies, and in particular, to an information processing method and apparatus, an electronic device, and an intelligent transportation system.

background

Video analysis technology is widely applied to various industries, and intelligent traffic systems based on video analysis are rapidly developed. The intelligent traffic system acquires video images of traffic scenes through the front-end video equipment, analyzes the video images of the traffic scenes through the front-end or rear-end video analysis system, acquires the advancing tracks and states of vehicles, pedestrians and non-motor vehicles in the traffic scenes, takes a snapshot and obtains evidence of traffic illegal events according to related traffic event rules, and provides data support for the optimal management of traffic management departments.

the intersection scene has more and more obvious influence on the whole traffic smoothness due to the fact that the number and types of traffic participants are large, the traffic participants are in a complicated state, and the traffic participants are in a standard state or not, so that the red light violation behaviors of pedestrians and non-motor vehicles are captured and collected, and the traffic participant scene is more and more important for the standard of the traveling of the pedestrians and the non-motor vehicles.

Disclosure of Invention

In view of the above, the present application provides an information processing method, an information processing apparatus, an electronic device, and an intelligent transportation system.

specifically, the method is realized through the following technical scheme:

According to a first aspect of embodiments of the present application, there is provided an information processing method, including:

identifying the state of a pedestrian signal lamp in a video image based on a video detection mode;

Performing target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

Determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp;

and if the target exists, sending the face image of the target with the red light running behavior to a back-end server, and generating red light running evidence obtaining information aiming at the target by the back-end server.

optionally, the identifying the state of the pedestrian signal lamp in the video image based on the video detection mode, and before the performing the target detection on the video image based on the deep learning algorithm, further include:

Preprocessing a video image;

the state of the pedestrian signal lamp in the video image is identified based on the video detection mode, and the method comprises the following steps:

identifying the state of a pedestrian signal lamp in the preprocessed video image based on a video detection mode;

the target detection of the video image based on the deep learning algorithm comprises the following steps:

and carrying out target detection on the preprocessed video image based on a deep learning algorithm.

optionally, the preprocessing the video image includes one or more of the following processing:

noise reduction, filtering, format conversion, and down-sampling.

optionally, the determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp includes:

for any target in the video image, determining the action track information of the target according to the target detection data of the video image comprising the target;

And determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp.

Optionally, the determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp includes:

And if the pedestrian signal lamp is in a red light state, the target enters a pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing is greater than a preset distance threshold, and the target is determined to have a red light running behavior.

optionally, the determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal light further includes:

and if the pedestrian signal lamp is in a red light state, the target enters a pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing is smaller than or equal to the preset distance threshold, determining that the target has a red light running trend, and performing early warning prompt.

Optionally, the sending the face image of the target with the red light running behavior to the back-end server includes:

For any target with red light running behavior, respectively intercepting small images comprising the target from a preset number of video images comprising the target, and splicing the intercepted small images to obtain a spliced image;

performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

and performing image quality grading on each face image based on a deep learning algorithm, and sending the face image with the highest grade to a back-end server.

optionally, the performing image quality scoring on each face image based on the deep learning algorithm includes:

image quality scoring of the face image is performed according to one or more of the following parameters:

the human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

According to a second aspect of the embodiments of the present application, there is provided an information processing apparatus applied to a front-end video capture device in an intelligent transportation system, the apparatus including:

The signal lamp state detection unit is used for identifying the state of a pedestrian signal lamp in the video image based on a video detection mode;

The target detection unit is used for carrying out target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

The behavior judgment unit is used for determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp;

and the communication unit is used for sending the face image of the target with the red light running behavior to a back-end server if the target with the red light running behavior exists, and generating red light running evidence obtaining information aiming at the target by the back-end server.

optionally, the apparatus further comprises:

the preprocessing unit is used for preprocessing the video image;

The signal lamp state detection unit identifies the state of a pedestrian signal lamp in the preprocessed video image based on a video detection mode;

and the target detection unit is used for carrying out target detection on the preprocessed video image based on a deep learning algorithm.

optionally, the preprocessing unit is specifically configured to perform one or more of the following processing on the video image:

noise reduction, filtering, format conversion, and down-sampling.

optionally, the target detection unit is further configured to determine, for any target in the video image, action trajectory information of the target according to target detection data of the video image including the target;

The behavior determination unit is specifically configured to determine whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp.

Optionally, the behavior determination unit is specifically configured to determine that a red light running behavior exists in the target if the target enters the pedestrian crossing area and the distance between the target and the initial position of the pedestrian crossing is greater than a preset distance threshold when the status of the pedestrian signal lamp is a red light status.

Optionally, the behavior determination unit determines that the target has a red light running trend if the target enters the pedestrian crossing area and the distance between the target and the initial position of the pedestrian crossing is less than or equal to the preset distance threshold when the status of the pedestrian signal lamp is a red light status;

The device further comprises:

And the early warning prompting unit is used for giving early warning prompt when the target is determined to have the red light running trend.

Optionally, the apparatus further comprises:

the human face detection unit is used for respectively intercepting small images comprising the target from a preset number of video images comprising the target for any target with the red light running behavior, and splicing the intercepted small images to obtain a spliced image; performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

The human face image scoring unit is used for scoring the image quality of each human face image based on a deep learning algorithm;

the communication unit is specifically configured to send the face image with the highest score to a back-end server.

Optionally, the facial image scoring unit is specifically configured to score the facial image according to image quality of one or more of the following parameters:

The human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

according to a third aspect of the embodiments of the present application, there is provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

A memory for storing a computer program;

and the processor is used for realizing the information processing method when executing the program stored in the memory.

according to a fourth aspect of the embodiments of the present application, an intelligent transportation system is provided, which includes a front-end video acquisition device and a back-end server; wherein:

The video acquisition equipment is used for identifying the state of a pedestrian signal lamp in a video image based on a video detection mode;

The video acquisition equipment is also used for carrying out target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

The video acquisition equipment is also used for determining whether a target red light running behavior exists in a video image according to the target detection data and the state of the pedestrian signal lamp; if the target exists, sending the face image of the target with the red light running behavior to the back-end server;

and the back-end server is used for generating red light running evidence obtaining information aiming at the target when receiving the human face image of the target with the red light running behavior sent by the video acquisition equipment.

according to the information processing method, the state of the pedestrian signal lamp in the video image is identified based on the video detection mode, the target detection is carried out on the video image based on the deep learning algorithm, whether the target red light running behavior exists in the video image is further determined according to the target detection data and the state of the pedestrian signal lamp, if the target red light running behavior exists, the face image of the target with the red light running behavior is sent to the rear-end server, the rear-end server generates the red light running evidence obtaining information aiming at the target, the detection rate and the efficiency of the detection of the red light running behavior are improved, and the application scene of the scheme is expanded.

drawings

Fig. 1 is an architecture diagram of an intelligent transportation system according to an exemplary embodiment of the present application;

FIG. 2 is a flow chart illustrating a method of information processing according to an exemplary embodiment of the present application;

FIG. 3 is a diagram illustrating a specific application scenario according to an exemplary embodiment of the present application;

Fig. 4A is a schematic structural diagram of a smart camera according to an exemplary embodiment of the present application;

FIG. 4B is a block diagram of a GPU according to an exemplary embodiment of the present disclosure;

FIG. 4C is a block diagram of a CPU shown in an exemplary embodiment of the present application;

Fig. 5 is a schematic structural diagram of an information processing apparatus shown in an exemplary embodiment of the present application;

fig. 6 is a schematic configuration diagram of an information processing apparatus shown in still another exemplary embodiment of the present application;

Fig. 7 is a schematic configuration diagram of an information processing apparatus shown in still another exemplary embodiment of the present application;

Fig. 8 is a schematic configuration diagram of an information processing apparatus shown in still another exemplary embodiment of the present application;

fig. 9 is a schematic diagram of a hardware structure of an electronic device according to an exemplary embodiment of the present application.

Detailed Description

reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

In order to make those skilled in the art better understand the technical solutions provided by the embodiments of the present application, a brief description is first given below of a system architecture to which the present application is applicable.

Referring to fig. 1, a schematic architecture diagram of an intelligent transportation system provided in an embodiment of the present application is shown in fig. 1, where the intelligent transportation system includes a front-end video capture device and a back-end server connected via a network, where:

the front-end video acquisition equipment can acquire video images of pedestrian crossing areas, wherein pedestrians, non-motor vehicles, motor vehicles and the like passing through the pedestrian crossing areas can be included in the view field of the front-end video acquisition equipment, and pedestrian signal lamps deployed in the pedestrian crossing areas.

on one hand, the front-end video acquisition equipment can identify the state of a pedestrian signal lamp in the acquired video image based on a video detection mode; the states of the pedestrian signal lamp can comprise a red light state, a green light state or a yellow light state.

On the other hand, the front-end video acquisition equipment can perform target detection on the acquired video image based on a deep learning algorithm to obtain target detection data; wherein the target may comprise a pedestrian or a non-motor vehicle.

the front-end equipment can determine whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp, and when the target red light running behavior exists, the front-end equipment sends the face image of the target with the red light running behavior to the rear-end server, and the rear-end server generates red light running evidence obtaining data for the target.

it should be noted that fig. 1 only shows a connection relationship between the front-end video capture device and the back-end server, but a specific connection manner between the front-end video capture device and the back-end server is not shown, for example, the front-end video capture device and the back-end server may be connected through a network switch, a network cable, and the like.

In order to make the aforementioned objects, features and advantages of the embodiments of the present application more comprehensible, embodiments of the present application are described in detail below with reference to the accompanying drawings.

Referring to fig. 2, a schematic flow chart of an information processing method provided in an embodiment of the present application is shown, where the information processing method may be applied to a front-end video capture device (hereinafter, an intelligent camera is taken as an example) in an intelligent transportation system, and as shown in fig. 2, the method may include the following steps:

And S200, identifying the state of the pedestrian signal lamp in the video image based on a video detection mode.

in the embodiment of the application, for each frame of video image collected by the intelligent camera, the intelligent camera can identify the state of the pedestrian signal lamp in the video image based on a video detection mode so as to determine that the current pedestrian signal lamp is in a red light state, a green light state or a yellow light state.

s210, performing target detection on the video image based on a deep learning algorithm to obtain target detection data; wherein the object comprises a pedestrian or a non-motor vehicle.

In the embodiment of the application, for each frame of video image collected by the intelligent camera, the intelligent camera can perform target detection on the video image based on a deep learning algorithm so as to detect pedestrians or non-motor vehicles existing in the video image.

it should be noted that, when the intelligent camera detects the target of the video image based on the deep learning algorithm, the head-shoulder model may be used for detection, and the specific implementation thereof is not described herein again.

It should be noted that, in the embodiment of the present application, there is no necessary timing relationship between step S200 and step S210, that is, the operation in step S200 may be executed first and then the operation in step S210 may be executed according to the description in the above method flow; alternatively, the operation in step S210 may be performed first, and then the operation in step S200 may be performed; alternatively, the operations in step S200 and step S210 may also be performed concurrently.

in an example, in order to reduce the load of the smart camera, the smart camera may perform target detection when detecting that the pedestrian signal light is in a red light state, and the specific implementation thereof is not described herein again.

in an embodiment of the application, before the performing the target detection on the video image based on the deep learning algorithm and before the performing the target detection on the video image based on the deep learning algorithm, the method may further include:

Preprocessing a video image;

accordingly, the identifying the state of the pedestrian signal lamp in the video image based on the video detection mode may include:

Identifying the state of a pedestrian signal lamp in the preprocessed video image based on a video detection mode;

the above target detection of the video image based on the deep learning algorithm may include:

And carrying out target detection on the preprocessed video image based on a deep learning algorithm.

In this embodiment, for each frame of video image captured by the smart camera, the smart camera may pre-process the video image before performing object detection and pedestrian signal light state identification.

For example, a smart camera may perform one or more of the following on the video image:

noise reduction, filtering, format conversion, and down-sampling.

specifically, in order to improve the accuracy of pedestrian signal lamp state identification and target detection and reduce the performance requirements of pedestrian signal lamp state identification and target detection, after the intelligent camera collects a video image and before pedestrian signal lamp state identification and target detection are carried out, on one hand, noise interference can be reduced through noise reduction and filtering processing so as to improve the accuracy of target detection; on the other hand, the video image can be converted into a format which can be processed when the intelligent camera carries out target detection through format conversion processing, and then the resolution of the video image is reduced through down-sampling processing, so that the size of the video image is reduced, and the performance requirement of target detection on the intelligent camera is reduced.

in this embodiment, after the video image is preprocessed by the smart camera, the state of the pedestrian signal lamp in the preprocessed video image can be identified based on a video detection mode, and the target detection is performed on the preprocessed video image based on a deep learning algorithm.

And S220, determining whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp. If yes, go to step S230; otherwise, the current flow is ended.

In the embodiment of the application, the intelligent camera can determine whether a target red light running behavior exists in the video image according to the target detection data and the state of the pedestrian signal lamp, namely whether a pedestrian or a non-motor vehicle runs the red light exists.

In an embodiment of the present application, the determining whether a target red light running behavior exists in a video image according to the target detection data and the state of the pedestrian signal light includes:

For any target in the video images, determining the action track information of the target according to target detection data of a continuous preset number of video images including the target;

and determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp.

in this embodiment, for any target in any frame of video image captured by the smart camera, the smart camera may determine a position change condition of the target in the video image according to target detection data of the video image including the target, and further determine action trajectory information of the target.

After the intelligent camera determines the action track information of the target, whether the red light running behavior exists in the target can be determined according to the action track information of the target and the state of the pedestrian signal lamp.

In an implementation manner of this embodiment, the determining whether the target has a red light running behavior according to the action track information of the target and the state of the pedestrian signal lamp includes:

and if the pedestrian signal lamp is in a red light state, the target enters the pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing area is greater than a preset distance threshold value, determining that the red light running behavior of the target exists.

In this embodiment, for any target, after the smart camera determines the action track information of the target, it may determine whether the target enters the pedestrian crossing area in a red light state and whether the distance between the target and the start position of the pedestrian crossing is greater than a preset distance threshold (which may be set according to an actual scene, such as half of the total length of the pedestrian crossing area) according to the action track information of the target and the real-time state of the pedestrian signal; if the target enters the crosswalk area in the red light state and the distance between the target and the initial position of the crosswalk is larger than the preset distance threshold, determining that the red light running behavior of the target exists.

The target enters the pedestrian crossing area from the outside of the pedestrian crossing area after the pedestrian signal lamp is switched from the yellow lamp state to the red lamp state; when the intelligent camera determines whether the distance between the target and the initial position of the pedestrian crossing is greater than a preset distance threshold, the detected target can be compared with the maximum distance between the target and the initial position of the pedestrian crossing and the preset distance threshold after entering the pedestrian crossing area in a red light state, and if the maximum distance is greater than the preset distance threshold, the distance between the target and the initial position of the pedestrian crossing is determined to be greater than a preset distance threshold.

further, in this embodiment, in consideration of the fact that the target does not notice the red light or does not realize that the target is going to run the red light when the target enters the crosswalk area in the red light state, the smart camera may automatically detect whether the target has a red light running trend, and perform an early warning prompt on the target having the red light running trend.

accordingly, in this embodiment, after determining the action track information of the target according to the target detection data of the video image including the target, the method may further include:

And if the pedestrian signal lamp is in a red light state, the target enters a pedestrian crossing area, and the distance between the target and the initial position of the pedestrian crossing is smaller than or equal to the preset distance threshold, determining that the target has a red light running trend, and performing early warning prompt.

specifically, when the intelligent camera detects that the target enters the pedestrian crossing area in the red light state, but the distance between the target and the initial position of the pedestrian crossing is smaller than or equal to a preset distance threshold value, the intelligent camera can determine that the target has a red light running trend, and at the moment, the intelligent camera can perform early warning prompt. For example, the smart camera may output an early warning prompt message such as "please do not run a red light" through a voice speaker or/and a display screen.

When the early warning prompt display screen exists, the intelligent camera can also display the image of the target with the red light running trend in the display screen.

It should be noted that, in the embodiment of the present application, the implementation manner of determining whether the target has the red light running behavior is only a specific example of determining whether the target has the red light running behavior by the smart camera, and is not limited to the scope of the present application.

And step S230, sending the face image of the target with the red light running behavior to a back-end server, and generating red light running evidence obtaining information aiming at the target by the back-end server.

In the embodiment of the application, after the intelligent camera determines that the target red light running behavior exists, a face image (such as a face close-up image) of the target with the red light running behavior can be acquired, and the face image is sent to the back-end server.

When the target is a non-motor vehicle, the face image of the target can refer to the face image of a non-motor vehicle driver.

For example, the smart camera may send a red light running notification message carrying the face image to the back-end server.

After receiving the face image of the target with the red light running behavior sent by the intelligent camera, the back-end server may match the face image in a third-party face database (e.g., a traffic system face database or a public security system face database) to determine identity information of the target (e.g., driver license information matched with the face image or identity information of the target included in the identity information), and generate red light running evidence obtaining information.

the red light running evidence obtaining information may include, but is not limited to, a name, an identification number, a red light running time (which may be a time of receiving a face image sent by a smart camera), a red light running position (which may be a deployment position of the smart camera sending the face image), a preset number of video images or videos in the red light running process, and the like of a target.

In the embodiment of the application, in consideration of the influence of the angle and the scale of the face in the face image on the accuracy of the face matching algorithm, after the intelligent camera determines that the red light running behavior exists in the target, the intelligent camera can respectively perform face detection on multiple frames of video images including the target aiming at the target to obtain multiple face images of the target, perform image quality grading on the multiple face images, and further take the face image with the highest grade as the face image for face matching and send the face image to the back-end server.

Accordingly, in one embodiment of the present application, the sending the face image of the target with the red light running behavior to the back-end server may include:

For any target with red light running behavior, respectively intercepting small images comprising the target from continuous preset number of video images comprising the target, and splicing the intercepted small images to obtain a spliced image;

Performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

and performing image quality grading on each face image based on a deep learning algorithm, and sending the face image with the highest grade to a back-end server.

In this embodiment, after the intelligent camera determines that there is a target with a red light running behavior, for any target with a red light running behavior, the intelligent camera may respectively capture the thumbnails of the target from the preset number (which may be set according to an actual scene, such as 5 frames, 10 frames, and the like) of video images including the target, and splice the captured thumbnails to obtain a spliced image of the target.

wherein, the thumbnail refers to an image of a region including a specified target cut out from one frame of video image.

after the target mosaic is obtained, the intelligent camera can perform face detection on the mosaic based on a deep learning algorithm to obtain a face image corresponding to each small image, and perform image quality grading on each face image based on the deep learning algorithm.

in one example, a smart camera may score the image quality of a facial image according to one or more of the following parameters:

The human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

in this embodiment, after the intelligent camera obtains the scores of the face images, the face image with the highest score can be sent to the back-end server, and the back-end server performs face matching according to the face image with the highest score, so that the accuracy of face matching is improved, and further, the reliability of red light running evidence obtaining data is improved.

It should be noted that, in this embodiment, in order to implement face image scoring, the face scoring parameters may be used to perform sample calibration for deep learning and face image scoring model training in advance, and the trained face scoring model is used to perform image quality scoring on the face image, which is not described herein in detail.

as can be seen, in the method flow shown in fig. 2, the detection rate and the efficiency of target detection are improved by performing target detection on the video image based on the deep learning algorithm, so that the detection rate and the efficiency of red light running behavior detection are improved; in addition, the states of pedestrian signal lamps in the video images are identified based on a video detection mode, an external signal lamp signal machine is not needed to provide traffic light signals, hardware equipment is reduced, the application cost of the scheme is reduced, and the application scene of the scheme is expanded; moreover, by introducing a face scoring mechanism, the accuracy of face matching is improved, the accuracy of identity recognition is correspondingly improved, and the reliability of red light running evidence obtaining data is further improved.

In order to enable those skilled in the art to better understand the technical solutions provided in the embodiments of the present application, the following describes the technical solutions provided in the embodiments of the present application with reference to specific application scenarios.

Referring to fig. 3, a schematic diagram of an application scenario provided in the embodiment of the present application is shown in fig. 3, where the application scenario includes a backend server, a smart camera (micro camera), a voice speaker or/and a display screen.

The intelligent camera collects video images of pedestrian crossing areas of the intersection in real time, and a processor embedded in the intelligent camera is used for analyzing and processing the video images in real time; for a suspected target that a pedestrian runs the red light in the visual field, the intelligent camera sends out an early warning signal and transmits the early warning signal out through a network; and for the target which still continues to run the red light after early warning, caching the picture or video segment in the process of running the red light, extracting the most appropriate human face image, and transmitting the most appropriate human face image to a back-end server through a network.

And the rear-end server receives the violation pictures and the video clips transmitted by the front-end camera, compares the face close-up picture with a third-party face database, determines the identity of the pedestrian running the red light target, forms complete violation evidence obtaining data and stores the data.

the voice prompt loudspeaker or the display screen receives prompt signals transmitted by the intelligent camera and broadcasts safe street-crossing prompt information in a circulating mode.

fig. 4A shows a schematic structural diagram of the smart camera in the application scenario shown in fig. 3, where the smart camera includes a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), a CCD (Charge-coupled Device), a DDR (Double Data Rate) memory module, and a communication module; the GPU, the CPU, the CCD, the DDR memory module and the communication module can communicate with each other through a system bus.

the CCD collects intersection scene images and outputs the collected image data to the DDR storage module.

the GPU reads a real-time image of a scene from the DDR, performs image preprocessing and caches the image to the DDR, wherein the image preprocessing comprises image format conversion, down-sampling processing and the like; and the pedestrian detection is carried out by adopting a deep learning algorithm, and the pedestrian detection data is transmitted to the DDR storage module.

The CPU reads real-time images and pedestrian detection data from the DDR, establishes corresponding pedestrian targets by combining historical pedestrian detection data, and tracks the pedestrian targets; the target track information formed by tracking is utilized, and the signal lamp state information obtained by video signal lamp detection is combined to analyze whether the red light running trend or illegal behaviors of the pedestrian exist or not; for a suspected target with a red light running trend, giving an early warning prompt signal mark and transmitting the mark to the DDR; and for the target with the red light violation, giving a violation confirming signal and relevant information of the violation target, and caching the corresponding process picture and the time point into the DDR.

and the GPU reads the pedestrian red light violation confirmation signal from the DDR, under the condition of confirming the violation, a plurality of frames of corresponding pedestrian target small images are intercepted from the real-time image by utilizing violation target position information for splicing, the spliced image is subjected to face detection and face grading of deep learning, and the most appropriate face image is selected and transmitted to the DDR storage module.

referring to fig. 4B and 4C, in this embodiment, the GPU may include an image preprocessing module, a pedestrian target detection module, a face detection module, and a face scoring module; the CPU may include a pedestrian target tracking module, a signal light detection module, and an event analysis module.

In this embodiment, taking the detection of red light running by a pedestrian as an example, the specific implementation flow is as follows:

the intelligent camera can acquire real-time images of the intersection scene through the CCD and store the acquired real-time image data to the DDR storage module.

The image preprocessing module of the GPU may read a real-time image of a scene from the DDR memory module, perform image preprocessing, and store the image preprocessed image into the DDR, where the image preprocessing may include noise reduction, filtering, image format conversion, down-sampling processing, and the like.

The pedestrian target detection module of the GPU can read the preprocessed real-time image from the DDR storage module, adopts a deep learning algorithm to detect pedestrians, obtains pedestrian detection data including pedestrian positions and pedestrian categories (normal walking, non-motor vehicles, pedestrians in motor vehicles or pedestrians printed on vehicle bodies and the like), and stores the pedestrian detection data in the DDR storage module.

The pedestrian target tracking module of the CPU can read real-time images and pedestrian detection data from the DDR storage module, establish corresponding pedestrian targets by combining historical pedestrian detection data, track the pedestrian targets, determine pedestrian target track information and store the pedestrian target track information to the DDR storage module.

the signal lamp detection module of the CPU can read the preprocessed real-time image from the DDR storage module, detect and classify the signal lamp, determine the state of the pedestrian signal lamp, and store the state information of the pedestrian signal lamp to the DDR storage module.

The event analysis module of the CPU can read the track information and the signal lamp state information of the pedestrian target from the DDR storage module, and analyzes whether the current pedestrian target has a red light running trend or a red light running behavior or not by combining the rule of the pedestrian red light running event.

For pedestrian targets with a red light running trend, outputting an early warning prompt signal mark to the DDR storage module; and for the pedestrian target with the red light running behavior, giving a violation confirmation signal and related information of the violation target, and storing a corresponding process picture and a corresponding time point to the DDR storage module.

the face detection module of the GPU can read violation confirmation signals from the DDR storage module and intercept a small pedestrian target image aiming at the pedestrian target which is confirmed to run the red light; after a plurality of frames of target pedestrian small images are continuously collected, the small images are spliced, face detection and position association are carried out on the spliced images, a face image of each small image corresponding to a pedestrian target is obtained, and the face images and related sequence information are stored in a DDR storage module.

The face scoring module of the GPU can read sequence face images from the DDR storage module, image quality scoring is carried out on each face image according to parameters such as face size, face definition, positive and negative face attributes, shielding proportion, face horizontal angle and face pitching angle, the face image with the highest score is selected as a face close-up image and stored in the DDR storage module.

The communication module can read the face close-up image from the DDR storage module, transmit the face close-up image to the back-end server, and perform face matching by the back-end server to determine identity information of the pedestrian target with the red light running behavior.

the communication module can also transmit the process picture or/and the video clip of the pedestrian target with the red light running behavior to the back-end server, and the back-end server generates the red light running evidence obtaining information according to the process picture or/and the video clip and the identity information.

The communication module can also read the early warning prompt signal sign from the DDR storage module, and transmits the early warning prompt signal sign to the voice loudspeaker or/and the display screen, and the voice loudspeaker or/and the display screen circularly broadcasts the safe street crossing prompt information.

In the embodiment of the application, the state of the pedestrian signal lamp in the video image is identified based on a video detection mode, the target detection is carried out on the video image based on a deep learning algorithm, then whether the target red light running behavior exists in the video image is determined according to the target detection data and the state of the pedestrian signal lamp, if the target red light running behavior exists, the face image of the target with the red light running behavior is sent to the rear-end server, the rear-end server generates red light running evidence obtaining information aiming at the target, the detection rate and the efficiency of the detection of the red light running behavior are improved, and the application scene of the scheme is expanded.

the methods provided herein are described above. The following describes the apparatus provided in the present application:

Referring to fig. 5, a schematic structural diagram of an information processing apparatus according to an embodiment of the present application is shown, where the information processing apparatus may be applied to a front-end video capture device in the intelligent transportation system shown in fig. 1, and as shown in fig. 5, the information processing apparatus may include:

A signal lamp state detection unit 510, configured to identify a state of a pedestrian signal lamp in the video image based on a video detection manner;

The target detection unit 520 is configured to perform target detection on the video image based on a deep learning algorithm to obtain target detection data; the target comprises a pedestrian or a non-motor vehicle;

A behavior determination unit 530, configured to determine whether a target red light running behavior exists in a video image according to the target detection data and the state of the pedestrian signal light;

and the communication unit 540 is configured to, if there is a red light running behavior of the target, send the face image of the target with the red light running behavior to the back-end server, and generate red light running evidence obtaining information for the target by the back-end server.

In an alternative embodiment, as shown in fig. 6, the apparatus further comprises:

a preprocessing unit 550, configured to preprocess the video image;

the signal lamp state detection unit 510 identifies the state of the pedestrian signal lamp in the preprocessed video image based on a video detection mode;

The target detection unit 520 performs target detection on the preprocessed video image based on a deep learning algorithm.

in an alternative embodiment, the pre-processing unit 550 is specifically configured to perform one or more of the following processes on the video image:

Noise reduction, filtering, format conversion, and down-sampling.

In an optional implementation manner, the object detection unit 520 is further configured to, for any object in the video image, determine the action trajectory information of the object according to the object detection data of the video image including the object;

the behavior determining unit 530 is specifically configured to determine whether a red light running behavior exists in the target according to the action trajectory information of the target and the state of the pedestrian signal lamp.

in an optional implementation manner, the behavior determining unit 530 is specifically configured to determine that a red light running behavior exists in the target if the target enters the pedestrian crossing area and the distance between the target and the start position of the pedestrian crossing is greater than a preset distance threshold when the status of the pedestrian signal is a red light status.

In an optional implementation manner, the behavior determination unit 530 determines that the target has a red light running trend if the target enters the pedestrian crossing area and the distance between the target and the start position of the pedestrian crossing is less than or equal to the preset distance threshold when the status of the pedestrian signal lamp is a red light status;

As shown in fig. 7, the apparatus further includes:

and the early warning prompting unit 560 is used for giving an early warning prompt when the target is determined to have a red light running trend.

in an alternative embodiment, as shown in fig. 8, the apparatus further comprises:

the face detection unit 570 is configured to, for any target with a red light running behavior, respectively intercept small images including the target from a preset number of video images including the target, and splice the intercepted small images to obtain a spliced image; performing face detection on the spliced image based on a deep learning algorithm to obtain a face image corresponding to each small image;

A face image scoring unit 580 for scoring the image quality of each face image based on a deep learning algorithm;

the communication unit 540 is specifically configured to send the face image with the highest score to a back-end server.

In an optional implementation manner, the facial image scoring unit 580 is specifically configured to score the facial image according to one or more of the following parameters:

the human face size, the human face definition, the positive and negative face attributes, the shielding proportion, the human face horizontal angle and the human face pitching angle.

Fig. 9 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present disclosure. The electronic device may include a processor 901, a communication interface 902, a memory 903, and a communication bus 904. The processor 901, the communication interface 902, and the memory 903 communicate with each other via a communication bus 904. Wherein, the memory 903 is stored with a computer program; the processor 901 can execute the information processing method described above by executing the program stored on the memory 903.

the memory 903 referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the memory 902 may be: RAM (random access memory), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, dvd, etc.), or similar storage medium, or a combination thereof.

Embodiments of the present application also provide a machine-readable storage medium, such as the memory 903 in fig. 9, storing a computer program, which can be executed by the processor 901 in the electronic device shown in fig. 9 to implement the information processing method described above.

it is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.