阅读说明：本技术 用于网格状路网的交通信号管控系统和设备 (Traffic signal management and control system and equipment for grid-shaped road network ) 是由 金峻臣 华文 汪作为 于 2021-06-22 设计创作，主要内容包括：本发明实施例公开了用于网格状路网的交通信号管控系统和设备,包括多个交通管控智能体,网格状路网包括多个交叉路口,网格状路网设置有道路检测设备和道路信号设备,交通管控智能体与交叉路口一一对应设置；交通管控智能体单向串联组成节点链条,节点链条中相邻的两个交通管控智能体对应相邻的两个交叉路口；交通管控智能体用于根据节点链条中上游的交通管控智能体的动作计算结果,以及对应的交叉路口的道路检测设备检测到的交通数据,基于预训练的模型,实时生成对应的交叉路口的信号控制指令,并将信号控制指令发送到道路控制设备。本方案减少交通信号管控优化过程中深度学习的动作和状态空间搜索,克服集中式控制中维度过大的问题。(The embodiment of the invention discloses a traffic signal control system and equipment for a latticed road network, which comprise a plurality of traffic control intelligent bodies, wherein the latticed road network comprises a plurality of intersections, the latticed road network is provided with road detection equipment and road signal equipment, and the traffic control intelligent bodies and the intersections are arranged in a one-to-one correspondence manner; the traffic control intelligent bodies are connected in series in one way to form a node chain, and two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections; and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on the pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data detected by the road detection equipment of the corresponding intersection, and sending the signal control instruction to the road control equipment. According to the scheme, the action and state space search of deep learning in the traffic signal control optimization process is reduced, and the problem of overlarge dimensionality in centralized control is solved.)

1. A traffic signal control system for a grid-shaped road network is characterized by comprising a plurality of traffic control intelligent bodies, wherein the grid-shaped road network comprises a plurality of intersections, the grid-shaped road network is provided with road detection equipment and road signal equipment, the traffic control intelligent bodies and the intersections are arranged in a one-to-one correspondence mode, the road detection equipment is used for detecting traffic data, and the road signal equipment is used for indicating the traveling states of the intersections;

the traffic control intelligent bodies are connected in series in one direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, and the traffic control intelligent bodies are connected with road detection equipment and road control equipment of the corresponding intersections;

and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data of the corresponding intersection, and sending the signal control instruction to the road control equipment.

2. The traffic signal management and control system according to claim 1, wherein the pre-trained models include an action calculation model and a state evaluation model;

the action calculation model is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to an action calculation result of an upstream traffic control agent in the node chain and traffic data of the corresponding intersection;

the state evaluation model is used for evaluating the influence of the signal control command on the traffic data and feeding the evaluation back to the action calculation model so as to update the action calculation model.

3. The traffic signal management and control system of claim 2, wherein the action calculation model comprises a recursive network layer;

the recursive network layer is used for establishing the correlation of the traffic control intelligent agent at the upstream in the node chain to the current traffic control intelligent agent.

4. The traffic signal management and control system according to claim 2, wherein the motion calculation model confirms the signal control command by learning the minimum loss value.

5. The traffic signal management and control system according to claim 2, wherein the state evaluation model evaluates the signal control command by calculating a value function.

6. The traffic signal management and control system according to claim 1, wherein traffic management and control agents corresponding to intersections in roads in a preset first direction in the grid-shaped road network are sequentially connected in series in a unidirectional manner.

7. The traffic signal management and control system according to claim 1, wherein the road detection device includes a camera, and the traffic data includes road image data.

8. The traffic signal management system of claim 1, wherein the road detection device comprises a radar and the traffic data comprises vehicle speed data.

9. The traffic signal management and control system of claim 1, wherein the road signaling device comprises a signal light and a guide screen.

10. Traffic signal management and control device for grid-like road network, characterized in that said traffic signal management and control device is integrated with a traffic signal management and control system according to any of claims 1-9.

Technical Field

The embodiment of the invention relates to the technical field of public services, in particular to a traffic signal management and control system and equipment for a latticed road network.

Background

Traffic congestion is a major public service problem in cities nowadays, and not only can huge economic loss be caused, but also traffic accidents can be increased. Traffic signal management and control optimization is an important means for relieving urban traffic jam. The existing traffic signal management and control optimization is not only manually optimized according to actual conditions and experience, but also based on a reinforcement learning framework, and as a more advanced method, the traffic signal management and control optimization can be realized in real time. For example, in a common traffic control optimization method based on reinforcement learning, a reinforcement learning Agent obtains an environmental status (state) by observing traffic flow, and controls traffic lights by an output action (action) to reduce traffic congestion or achieve other targets, so as to obtain reward feedback (reward).

A Q table consisting of traffic states and actions is established by using a traditional reinforcement Learning traffic control optimization method, and the Q table is updated through algorithms such as Q-Learning and SARSA. The traditional method is more concentrated on management and control optimization control of a single intersection, is only suitable for discrete and low-dimensional state spaces, and is difficult to apply to a road network with a high-dimensional management and control scheme and under a complex traffic state.

Disclosure of Invention

The invention provides a traffic signal management and control system and equipment for a latticed road network, which aim to solve the technical problem that high-dimensional management and control are difficult to realize in a complex traffic state in the prior art.

In a first aspect, an embodiment of the present invention provides a traffic signal management and control system for a grid-like road network, including a plurality of traffic management and control agents, where the grid-like road network includes a plurality of intersections, the grid-like road network is provided with road detection devices and road signal devices, the traffic management and control agents and the intersections are arranged in a one-to-one correspondence manner, the road detection devices are used to detect traffic data, and the road signal devices are used to indicate the traveling states of the intersections;

the traffic control intelligent bodies are connected in series in one direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, and the traffic control intelligent bodies are connected with road detection equipment and road control equipment of the corresponding intersections;

and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data of the corresponding intersection, and sending the signal control instruction to the road control equipment.

Wherein the pre-trained model comprises an action calculation model and a state evaluation model;

the action calculation model is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to an action calculation result of an upstream traffic control agent in the node chain and traffic data of the corresponding intersection;

the state evaluation model is used for evaluating the influence of the signal control command on the traffic data and feeding the evaluation back to the action calculation model so as to update the action calculation model.

Wherein the action calculation model comprises a recursive network layer;

the recursive network layer is used for establishing the correlation of the traffic control intelligent agent at the upstream in the node chain to the current traffic control intelligent agent.

Wherein the motion calculation model confirms the signal control command by learning a minimization loss value.

Wherein the state evaluation model evaluates the signal control command by a calculation function.

The traffic control intelligent bodies corresponding to intersections in roads in a preset first direction in the latticed road network are sequentially connected in series in a one-way mode.

Wherein the road detection device comprises a camera and the traffic data comprises road image data.

Wherein the road detection device comprises a radar and the traffic data comprises vehicle speed data.

Wherein the road signaling device includes a signal light and a guide screen.

In a second aspect, an embodiment of the present invention provides a traffic signal management and control device for a mesh-like road network, where the traffic signal management and control device is integrated with any one of the traffic signal management and control systems of the first aspect.

The traffic signal control system and the traffic signal control equipment for the grid-shaped road network comprise a plurality of traffic control intelligent bodies, wherein the grid-shaped road network comprises a plurality of intersections, the grid-shaped road network is provided with road detection equipment and road signal equipment, the traffic control intelligent bodies and the intersections are arranged in a one-to-one correspondence manner, the road detection equipment is used for detecting traffic data, and the road signal equipment is used for indicating the traveling state of the intersections; the traffic control intelligent bodies are connected in series in one direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, and the traffic control intelligent bodies are connected with road detection equipment and road control equipment of the corresponding intersections; and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data of the corresponding intersection, and sending the signal control instruction to the road control equipment. The shared information of the adjacent intersections is established in a mode that the traffic control intelligent bodies corresponding to each intersection are connected in series into node chains, so that each traffic control intelligent body can observe the change of the whole latticed road network, particularly the change of the adjacent intersections, the action and state space search of deep learning in the traffic signal control optimization process is reduced, and the problem of overlarge dimensionality in centralized control is solved.

Drawings

FIG. 1 is a schematic structural diagram of a grid-like road network;

fig. 2 is a schematic structural diagram of a traffic signal management and control system for a mesh road network according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration and not limitation. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Fig. 1 is a schematic structural diagram of a grid network, and fig. 2 is a schematic structural diagram of a traffic signal management and control system for a grid network according to an embodiment of the present invention. As shown in the figure, the traffic signal control system for the grid-shaped road network comprises a plurality of traffic control intelligent bodies, wherein the grid-shaped road network comprises a plurality of intersections, the grid-shaped road network is provided with road detection equipment and road signal equipment, the traffic control intelligent bodies and the intersections are arranged in a one-to-one correspondence manner, the road detection equipment is used for detecting traffic data, and the road signal equipment is used for indicating the traveling states of the intersections;

the traffic control intelligent bodies are connected in series in one direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, and the traffic control intelligent bodies are connected with road detection equipment and road control equipment of the corresponding intersections;

and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data of the corresponding intersection, and sending the signal control instruction to the road control equipment.

For a traffic control agent, a pre-trained model is mainly trained in a reinforcement learning mode, a high-dimensional control scheme and the actual characteristics of a complex traffic state are provided for a latticed road network, a deep neural network is introduced to approximate a high-dimensional state space in the reinforcement learning, and the state space can be represented and effectively searched through information of a learning environment. In the prior art, each intersection independently applies a deep reinforcement learning method, and the control optimization of each intersection can improve the traffic condition of one intersection or a plurality of adjacent intersections, but as the scale of the road network increases, the traffic control intelligent bodies are difficult to learn in a huge state space, and the discrete traffic control intelligent bodies cannot observe the state of the whole latticed road network, so that the effective control of the whole latticed road network cannot be realized.

In the scheme, a reinforcement learning-based framework is further combined with a deep neural network, the states and action spaces of complex intersections in a grid-shaped road network are decomposed, each traffic control intelligent body can only generate a signal control instruction for one intersection, but all traffic control intelligent bodies are connected in series in a single direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, through the design, the state-action search space under the combined action of the traffic control intelligent bodies can be reduced, the dimension of the state space can be reduced along with the reduction, the training speed and the stability are improved under the condition of not sacrificing the training performance, meanwhile, the value function of the traffic control intelligent bodies can be corrected in the training process, and the method is more suitable for the application scene of traffic control.

As for the actual grid-shaped road network, the actual grid-shaped road network can be abstracted into the road network structure shown in fig. 1, that is, in both the vertical and horizontal directions, there are a plurality of roads, and the intersection point of the roads is the intersection point. Corresponding to the road network structure manner shown in fig. 1, when node chains are specifically implemented, traffic control agents corresponding to intersections in roads in a preset first direction in the latticed road network are sequentially connected in series in a unidirectional manner. Specifically, as shown in fig. 2, a first direction is determined (a transverse direction is taken as the first direction in fig. 2), traffic control intelligent bodies corresponding to intersections of roads in the first direction are sequentially connected in a one-way manner, then traffic control intelligent bodies corresponding to end points of two parallel roads are connected, and finally, the structural schematic diagram of the node chain shown in fig. 2 is formed. Of course, in a concrete grid-shaped road network, an abstraction as shown in fig. 1 may not be realized due to the direction, length, etc., but it is guaranteed that an approximate processing manner is adopted in the adjacent area as a whole, and the traffic state of one place does not substantially affect the traffic state of the other place when the two places are far away from each other for the road arrangement. Therefore, only the approximate processing mode is adopted for the adjacent areas, the influence relation between the single intersection and the whole road network can still be accurately described, and an accurate signal control command is generated and correspondingly evaluated.

In addition, the traffic control intelligent bodies corresponding to the intersections in the scheme can be trained respectively, distributed respectively and operated cooperatively, namely, each intersection is provided with the traffic control intelligent bodies with distinct hardware main bodies for traffic signal control, or the traffic control intelligent bodies corresponding to a plurality of intersections operate in one or more hardware main bodies in an integrated mode, and the traffic control intelligent bodies do not have obvious physical boundaries.

In a specific implementation process, the pre-trained model comprises an action calculation model and a state evaluation model;

the action calculation model is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to an action calculation result of an upstream traffic control agent in the node chain and traffic data of the corresponding intersection;

the state evaluation model is used for evaluating the influence of the signal control command on the traffic data and feeding the evaluation back to the action calculation model so as to update the action calculation model.

Further, the action calculation model comprises a recursive network layer;

the recursive network layer is used for establishing the correlation of the traffic control intelligent agent at the upstream in the node chain to the current traffic control intelligent agent.

In addition, the motion calculation model may confirm the signal control command by learning the minimum loss value. The state estimation model may evaluate the signal control command by a calculation function.

Referring to fig. 2, after receiving the traffic environment state(s), the traffic control agent correspondingly performs a control action (a), i.e. generates a signal control command. In the description of the present embodiment, for the purpose of differentiation description, six traffic control agents are defined as traffic control agents 1 and … and traffic control agent 6, and correspond to intersection 1, intersection … and intersection 6, respectively; traffic environment states corresponding to the traffic control agent are respectively defined as s1, … and s 6; the control actions correspondingly generated by the traffic control agent are respectively defined as a1, … and a 6. In the specific process of generating the control instruction, if the traffic control agent has the previous traffic control agent in the node chain, the control instruction of the previous traffic control agent needs to be input to perform comprehensive judgment. On the whole, the traffic control agent corresponding to each intersection reduces the search space of actions through the decision-making mode.

The training process of the model in the scheme follows the core architecture of an Actor-criticc algorithm in reinforcement learning. Specifically, the traffic control agent using the Actor-Critic algorithm is composed of two models, namely an action calculation model and a state evaluation model, which are respectively used for calculating an action and evaluating a state value through a state, wherein the action is used for controlling the action of the traffic control agent through learning an optimal strategy, and the final state is evaluated through a calculation value function. The action calculation model comprises a recursion network layer which is based on the structural design of a recursion neural network, specifically, in the scheme, the whole control decision problem of the grid-shaped road network is decomposed into sub-problems of a plurality of intersections, the processing difficulty is greatly reduced compared with the whole control decision of the whole grid-shaped road network, the traffic control intelligent body does not output the action set of all the intersections at one time, but continuously outputs the action of each intersection, and the huge strategy search space is effectively reduced. In the process of generating work, the traffic control intelligent agent fully considers the interaction between intersections, remembers preorder action decisions and explores the correlation between preorder actions. That is, for the control action ak of the traffic control agent k in the node chain, the control actions a1, …, ak output by the traffic control agent k in the preamble of the node chain are considered comprehensively.

In the traffic signal management and control system in the present solution, if the road detection device includes a camera, the corresponding traffic data includes road image data. If the road detection device comprises radar, the traffic data correspondingly comprises vehicle speed data.

If the road signal equipment comprises a signal lamp and a guide screen, the traffic control intelligent agent can simultaneously generate signal control instructions for the signal lamp and the guide screen, wherein the signal control instructions for the signal lamp are used for controlling color transformation of the signal lamp, and the signal control instructions for the guide screen are used for controlling display contents of the guide screen.

The traffic signal control system for the grid-shaped road network comprises a plurality of traffic control intelligent bodies, wherein the grid-shaped road network comprises a plurality of intersections, the grid-shaped road network is provided with road detection equipment and road signal equipment, the traffic control intelligent bodies and the intersections are arranged in a one-to-one correspondence manner, the road detection equipment is used for detecting traffic data, and the road signal equipment is used for indicating the traveling states of the intersections; the traffic control intelligent bodies are connected in series in one direction to form a node chain, two adjacent traffic control intelligent bodies in the node chain correspond to two adjacent intersections, and the traffic control intelligent bodies are connected with road detection equipment and road control equipment of the corresponding intersections; and the traffic control intelligent agent is used for generating a signal control instruction of the corresponding intersection in real time based on a pre-trained model according to the action calculation result of the traffic control intelligent agent at the upstream in the node chain and the traffic data of the corresponding intersection, and sending the signal control instruction to the road control equipment. The shared information of the adjacent intersections is established in a mode that the traffic control intelligent bodies corresponding to each intersection are connected in series into node chains, so that each traffic control intelligent body can observe the change of the whole latticed road network, particularly the change of the adjacent intersections, the action and state space search of deep learning in the traffic signal control optimization process is reduced, and the problem of overlarge dimensionality in centralized control is solved.

In the scheme, the invention further provides traffic signal management and control equipment for the grid-shaped road network, and the traffic signal management and control equipment is integrated with any one of the traffic signal management and control systems in the embodiments of the invention. In this scheme, can every traffic management and control agent correspond integrated terminal equipment respectively, also can a plurality of traffic management and control agents are integrated in a terminal equipment, and all traffic management and control agents are integrated in a terminal equipment even, but no matter which kind of integrated mode of adoption, must guarantee that all traffic management and control agents can generate the signal control instruction of the intersection that corresponds fast accurately, guarantee the unblocked operation of each intersection, and then guarantee the unblocked operation of whole latticed road network.

From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.