阅读说明：本技术 自动驾驶车辆无信号交叉口协调通行控制系统 (Coordination passing control system for signalless intersection of automatic driven vehicle ) 是由 崔建勋 曲明成 于 2021-08-19 设计创作，主要内容包括：自动驾驶车辆无信号交叉口协调通行控制系统,属于车辆自动驾驶技术领域。解决了自动驾驶车辆在无信号灯交叉口时安全性差的问题。本发明包括：车辆智能体VA和交叉口智能体IA；车辆智能体VA设置在无人驾驶车辆上,用于检测自身所在的无人驾驶车辆是否进入无灯交叉路口区域,若是进入无灯交叉路口区域,向对应的无灯交叉路口区域的交叉口智能体IA发送通过请求信号和车辆的实时状态信息；所述交叉口智能体IA设置在无灯交叉路口区域内,与发送通过请求信号的车辆智能体VA建立无线通信连接,并接收车辆智能体VA的实时状态信息；本发明适用于无人驾驶车辆通过无信号交叉路口。(A coordinated traffic control system for a signalless intersection of an automatically driven vehicle belongs to the technical field of automatic driving of vehicles. The problem of the security of automatic vehicle when no signal lamp crossing is poor is solved. The invention comprises the following steps: a vehicle agent VA and an intersection agent IA; the vehicle intelligent agent VA is arranged on the unmanned vehicle and is used for detecting whether the unmanned vehicle where the vehicle intelligent agent VA is located enters the lightless intersection area or not, and if the unmanned vehicle enters the lightless intersection area, the vehicle intelligent agent VA sends a passing request signal and real-time state information of the vehicle to the intersection intelligent agent IA of the corresponding lightless intersection area; the intersection intelligent agent IA is arranged in the area of the lightless intersection, establishes wireless communication connection with the vehicle intelligent agent VA which sends the passing request signal, and receives real-time state information of the vehicle intelligent agent VA; the invention is suitable for the unmanned vehicle to pass through the no-signal intersection.)

1. A coordinated traffic control system for an intersection where autonomous vehicles do not have a signal, comprising: a vehicle agent VA and an intersection agent IA;

the vehicle intelligent body VA is arranged on the unmanned vehicle and is used for detecting whether the unmanned vehicle where the vehicle intelligent body VA is located enters the lightless intersection area or not, if the unmanned vehicle enters the lightless intersection area, the vehicle intelligent body VA establishes wireless communication connection with the intersection intelligent body IA of the corresponding lightless intersection area, and sends a passing request signal and real-time state information of the vehicle to the intersection intelligent body IA;

the intersection intelligent agent IA is arranged in the area of the lightless intersection, establishes wireless communication connection with the vehicle intelligent agent VA which sends the passing request signal, and receives real-time state information of the vehicle intelligent agent VA;

the intersection intelligent agent IA is also used for determining the speed, acceleration and steering wheel turning angle control signals of each unmanned vehicle passing the request at the next moment by utilizing the fully-connected deep neural network trained in a deep Q learning mode according to the intersection geometric information, the traffic regulation information and the real-time state information of each unmanned vehicle passing the request, and sending the obtained control signals to the corresponding vehicle intelligent agent VA;

and the vehicle intelligent body VA is also used for forwarding the received speed, acceleration and steering wheel turning angle control signals to an automatic control system of the vehicle, so that the unmanned vehicle is controlled, and the updated real-time state information is sent to the intersection intelligent body IA until the unmanned vehicle drives out of the intersection area without the lamp.

2. The system of claim 1, wherein the real-time status information of the vehicle comprises: the current location, speed, direction of travel, vehicle width, and priority traffic class of the unmanned vehicle.

3. The coordinated traffic control system at an intersection where autonomous vehicles do not have a signal according to claim 2, wherein the priority traffic class of the vehicle is: the priority of the common vehicle is set to be constant 1, the priority of the public transport vehicle is set to be constant 3, and the priority of the emergency rescue vehicle is set to be constant 10.

4. The coordinated traffic control system at an intersection without signal for an autonomous vehicle as claimed in claim 1, wherein the inputs of the fully connected deep neural network are arranged according to time and priority traffic levels for each state vector requesting to pass through the autonomous vehicle to form a state matrix.

5. The coordinated traffic control system at an intersection of autonomous vehicles according to claim 4, wherein the state vector of each autonomous vehicle includes x and y coordinates of the current position, speed, width, length, passing direction and priority traffic class.

6. The system of claim 5, wherein the reward function for the deep Q learning mode is:

and r (S, a) is the real-time reward benefit of the unmanned vehicle to pass through adopting the decision action a in the state S, wherein a is the decision action and S is the real-time state of the unmanned vehicle.

7. The system of claim 6, wherein the fully-connected deep neural network process trained in a deep Q learning manner comprises:

step one, based on the epsilon-greedy strategy, giving a state conversion sample < s, a, r, s' >;

step two, calculating an expected total income Q value corresponding to the strategy action a under the current state s by using the state conversion sample < s, a, r, s' >;

step three, executing the strategic action a with the maximum Q value, converting the environment state of the automatic driving vehicle into a state s ', and calculating the expected total income Q value corresponding to the strategic action a ' in the state s ';

step four, taking < s ', a', r, s '> as a state transition sample < s, a, r, s' >, and returning to execute the step two until the expected total profit Q value converges.

8. The system for coordinated traffic control at an intersection where autonomous vehicles do not have a signal according to claim 7, wherein the calculation formula of the expected total profit Q corresponding to the strategic action a is as follows:

Q(s，a)＝r(s，a)+max_a′Q(s′，a′)

wherein Q (s, a) is the expected total benefit, max, of the selected policy action a in state s_a′Q (s ', a') is the maximum expected total benefit of the selected policy action a 'when the input state is s'.

9. The coordinated traffic control system at an intersection where autonomous vehicles do not have a signal as claimed in claim 1, wherein the intersection agent IA and the vehicle agent VA establish wireless communication through radio signals, GSM, WI-FI, 4G or 5G.

Technical Field

The invention belongs to the technical field of automatic driving of vehicles.

Background

The intersection without signal control relates to vehicles with different inlet directions, and the vehicles are required to pass through the intersection at the same time, so that potential collision is caused in certain areas within the range of the intersection, and the risk of collision accidents is increased. For human driving, in the case of the intersection of this type, human drivers can automatically make a "negotiation" between themselves to adjust their driving behaviors, and then sequentially pass through the intersection in order, which usually does not cause traffic accidents. However, for autonomous vehicles, the "potential conflict" traffic at non-signalized intersections constitutes a very complex traffic scenario, requiring a high level of intelligence to ensure safe traffic. In addition, a plurality of automatic driving vehicles at the non-signal intersection can finish high-efficiency and safe passage at a collective level only by effective coordination and communication, so that the negotiation and communication at the collective level cannot be finished well only by depending on the intelligent decision of the automatic driving vehicle monomer.

Disclosure of Invention

The invention aims to solve the problem that the safety of an automatic driving vehicle at a signalless intersection is poor, and provides a coordinated traffic control system for the signalless intersection of the automatic driving vehicle.

The invention relates to a signalless intersection coordinated passing control system for an automatic driving vehicle, which comprises a vehicle intelligent agent VA and an intersection intelligent agent IA;

the vehicle intelligent body VA is arranged on the unmanned vehicle and is used for detecting whether the unmanned vehicle where the vehicle intelligent body VA is located enters the lightless intersection area or not, if the unmanned vehicle enters the lightless intersection area, the vehicle intelligent body VA establishes wireless communication connection with the intersection intelligent body IA of the corresponding lightless intersection area, and sends a passing request signal and real-time state information of the vehicle to the intersection intelligent body IA;

the intersection intelligent agent IA is arranged in the area of the lightless intersection, establishes wireless communication connection with the vehicle intelligent agent VA which sends the passing request signal, and receives real-time state information of the vehicle intelligent agent VA;

the intersection intelligent agent IA is also used for determining the speed, acceleration and steering wheel turning angle control signals of each unmanned vehicle passing the request at the next moment by utilizing the fully-connected deep neural network trained in a deep Q learning mode according to the intersection geometric information, the traffic regulation information and the real-time state information of each unmanned vehicle passing the request, and sending the obtained control signals to the corresponding vehicle intelligent agent VA;

and the vehicle intelligent body VA is also used for forwarding the received speed, acceleration and steering wheel turning angle control signals to an automatic control system of the vehicle, so that the unmanned vehicle is controlled, and the updated real-time state information is sent to the intersection intelligent body IA until the unmanned vehicle drives out of the intersection area without the lamp.

Further, in the present invention, the real-time status information of the vehicle includes: the current location, speed, direction of travel, vehicle width, and priority traffic class of the unmanned vehicle.

Further, in the present invention, the priority traffic level of the vehicle: the priority of the common vehicle is set to be constant 1, the priority of the public transport vehicle is set to be constant 3, and the priority of the emergency rescue vehicle is set to be constant 10.

Further, in the invention, the input of the fully-connected deep neural network is that each request passes through the state vector of the automatic driving vehicle and is arranged according to time and priority traffic level to form a state matrix.

Further, in the present invention, the state vector of each autonomous vehicle includes an x-coordinate and a y-coordinate of a current position, a speed, a width, a length, a passing direction, and a priority traffic level.

Further, in the present invention, the reward function of the deep Q learning manner is:

and r (S, a) is the real-time reward benefit of the unmanned vehicle to pass through adopting the decision action a in the state S, wherein a is the decision action and S is the real-time state of the unmanned vehicle.

Further, in the present invention, the process of the fully-connected deep neural network trained in the deep Q learning manner is as follows:

step one, based on the epsilon-greedy strategy, giving a state conversion sample < s, a, r, s' >;

step two, calculating an expected total income Q value corresponding to the strategy action a under the current state s by using the state conversion sample < s, a, r, s' >;

step three, executing the strategic action a with the maximum Q value, converting the environment state of the automatic driving vehicle into a state s ', and calculating the expected total income Q value corresponding to the strategic action a ' in the state s ';

step four, taking < s ', a', r, s '> as a state transition sample < s, a, r, s' >, and returning to execute step two until the expected total gain Q value converges. Further, in the present invention, the calculation formula of the expected total profit Q value corresponding to the policy action a is:

Q(s，a)＝r(s，a)+max_a′Q(s′，a′)

wherein Q (s, a) is the expected total benefit, max, of the selected policy action a in state s_a′Q (s ', a') is the maximum expected total benefit of the selected policy action a 'when the input state is s'.

Further, in the invention, the intersection intelligent agent IA and the vehicle intelligent agent VA establish wireless communication through radio signals, GSM, WI-FI, 4G or 5G.

In the invention, each vehicle entering the intersection is provided with the intelligent internet communication equipment for communication with the intelligent agent IA at the intersection, and the communication signal needs to ensure the precision and the stability under high-speed movement. May be any of radio signals, GSM, WI-FI, 4G, 5G, etc. Each vehicle is an automatic drive-by-wire vehicle which can autonomously and unmanned control an accelerator, a steering wheel corner and braking.

The intersection intelligent agent IA is responsible for coordinating a plurality of vehicle intelligent agents VA requesting to enter the intersection for passing, stores geometrical information and traffic regulation information about the intersection in advance, and continuously acquires state information of the requesting vehicle through real-time communication for coordinating and controlling input of decisions. The output of which is to issue motion commands to each requesting vehicle. In this mode, the vehicle VA is completely controlled by the intersection IA, thereby achieving collective level coordinated operation of "central coordinated dispatching".

Drawings

FIG. 1 is a schematic diagram of a coordinated traffic control system for a signalless intersection of autonomous vehicles in accordance with the present invention;

FIG. 2 is a schematic diagram of a message passing mechanism of a vehicle agent VA and an intersection agent IA of the automatic vehicle signalless intersection coordinated traffic control system according to the present invention;

fig. 3 is a diagram of a Q estimation network architecture.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflict.

The first embodiment is as follows: the following describes an embodiment of the coordinated traffic control system for an automatic driving vehicle no-signal intersection, with reference to fig. 1 to 3, where the coordinated traffic control system includes: a vehicle agent VA and an intersection agent IA;

the vehicle intelligent body VA is arranged on the unmanned vehicle and is used for detecting whether the unmanned vehicle where the vehicle intelligent body VA is located enters the lightless intersection area or not, if the unmanned vehicle enters the lightless intersection area, the vehicle intelligent body VA establishes wireless communication connection with the intersection intelligent body IA of the corresponding lightless intersection area, and sends a passing request signal and real-time state information of the vehicle to the intersection intelligent body IA;

the intersection intelligent agent IA is arranged in the area of the lightless intersection, establishes wireless communication connection with the vehicle intelligent agent VA which sends the passing request signal, and correspondingly receives real-time state information of the vehicle intelligent agent VA;

the intersection intelligent agent IA is also used for determining the speed, acceleration and steering wheel turning angle control signals of each unmanned vehicle passing the request at the next moment by utilizing the fully-connected deep neural network trained in a deep Q learning mode according to the intersection geometric information, the traffic regulation information and the real-time state information of each unmanned vehicle passing the request, and sending the obtained control signals to the corresponding vehicle intelligent agent VA;

and the vehicle intelligent body VA is also used for forwarding the received speed, acceleration and steering wheel turning angle control signals to an automatic control system of the vehicle, so that the unmanned vehicle is controlled, and the updated real-time state information is sent to the intersection intelligent body IA until the head of the unmanned vehicle drives out of the intersection area without the lamp.

The communication mechanism between the vehicle intelligent agent VA and the intersection intelligent agent IA of the system is shown in FIG. 2, and the system has the following working flows:

(1) when a vehicle arrives at an intersection, the vehicle agent first establishes a communication link with the intersection agent, and the link is not cancelled until the vehicle exits the coordinated control area. The driving right of the intelligent networked automobile in the cooperative control area is completely controlled by an intelligent agent IA at the intersection;

(2) after receiving the request information from the vehicle agent, the intersection agent IA starts to input the information, such as the position, speed, passing direction and the like of the request agent, into the intersection agent IA together with the state information of other related intelligent vehicles in the cooperative control area, so as to generate response information and control decision for the intelligent vehicle. If no response message can be generated at this time, IA instructs the target VA to decelerate until a full stop awaits further instruction; (3) after the communication connection is established between the vehicle intelligent agent VA and the intersection intelligent agent IA, the unmanned vehicle control system completely authorizes the IA to control, including steering of a steering wheel, the size of an accelerator and displacement of a brake plate;

(4) after the smart vehicle passes through the intersection, the IA releases the communication link with the smart vehicle and then hands vehicle control back to the unmanned vehicle control system.

For example, the vehicle a/b./C/D simultaneously requires to pass through the intersection, the intersection intelligent agent IA selects a passing vehicle according to the time and priority of receiving the signal, determines the strategy to be executed by the vehicle a after receiving the vehicle a signal, sends the execution strategy signal to the vehicle a, and sends the current state to the intersection intelligent agent IA again after the vehicle a finishes acting.

Further, in the present embodiment, the real-time status information of the vehicle includes: the current location, speed, direction of travel, vehicle width, and priority traffic class of the unmanned vehicle.

The IA is responsible for receiving the passing requests of the intelligent agents VA of the vehicles entering the intersection, determining the control logic for coordinating the passing according to the state and the target of each request VA, the geometry of the intersection, the traffic rules and other conditions, and sending the control decision to each VA needing coordination in a response information mode.

Further, in the present embodiment, the priority traffic class is set according to whether the vehicle is an emergency vehicle or a normal vehicle, and the priority traffic class of the vehicle: the priority of the common vehicle is set to be constant 1, the priority of the public transport vehicle is set to be constant 3, and the priority of the emergency rescue vehicle is set to be constant 10.

Further, in the present embodiment, the input of the fully connected deep neural network is a state matrix formed by arranging state vectors of autonomous vehicles according to time and priority traffic levels for each request.

Further, in the present embodiment, the state vector of each autonomous vehicle includes an x coordinate and a y coordinate of a current position, a speed, a width, a length, a passing direction, and a priority traffic level.

Further, in the present embodiment, the reward function of the deep Q learning method is:

and r (S, a) is the real-time reward benefit of the unmanned vehicle to pass through adopting the decision action a in the state S, wherein a is the decision action and S is the real-time state of the unmanned vehicle.

In the present embodiment, when the speed is greater than 0 and no collision occurs, the benefit is proportional to the product of the speed and the priority; when the speed is equal to 0 but no collision occurs, this means that VA is forced to stop waiting, where the gain is small, 0.1; when a collision accident occurs, the profit is 0 at this time.

Further, in this embodiment, the process of the fully-connected deep neural network trained in the deep Q learning manner is as follows:

step one, based on the epsilon-greedy strategy, giving a state conversion sample < s, a, r, s' >;

step two, calculating an expected total income Q value corresponding to the strategy action a under the current state s by using the state conversion sample < s, a, r, s' >;

step three, executing the strategic action a with the maximum Q value, converting the environment state of the automatic driving vehicle into a state s ', and calculating the expected total income Q value corresponding to the strategic action a ' in the state s ';

step four, taking < s ', a', r, s '> as a state transition sample < s, a, r, s' >, and returning to execute step two until the expected total gain Q value converges.

In the real-time method, the Q estimation network architecture of IA is shown in fig. 3. The state input is all information of the currently needed cooperative decision, a network architecture adopts a fully-connected deep neural network, no less than 5 layers are suggested, and Q values corresponding to three actions are output. Respectively as follows: a is₁(maintaining the current speed), a₂(acceleration) and a₃(deceleration). Input requirements for the Q estimation network include requesting vehicle and current requirements for co-referenceAll status information of other vehicles considered. The status information for each smart vehicle is described by 7 dimensions, respectively position x coordinate, position y coordinate, speed, width, length, destination (direction), and priority traffic level. Assuming that the maximum number of vehicles that currently need to be coordinated is m, the entire state information is represented by a matrix of m × 7 dimensions, and in order to input the information into the Q estimation network, a straightening operation is further employed to elongate it into a 7 m-dimensional vector.

Further, in this embodiment, the calculation formula of the expected total profit Q value corresponding to the policy action a is as follows:

Q(s，a)＝r(s，a)+max_a′Q(s′，a′)

wherein Q (s, a) is the expected total benefit, max, of the selected policy action a in state s_a′Q (s ', a') is the maximum expected total benefit of the selected policy action a 'when the input state is s'.

Further, in this embodiment, the intersection agent IA and the vehicle agent VA establish wireless communication via radio signals, GSM, WI-FI, 4G, or 5G.

In the invention, each vehicle entering the intersection is provided with the intelligent internet communication equipment for communication with the intelligent agent IA at the intersection, and the communication signal needs to ensure the precision and the stability under high-speed movement. May be any of radio signals, GSM, WI-FI, 4G, 5G, etc. Each vehicle is an automatic drive-by-wire vehicle which can autonomously and unmanned control an accelerator, a steering wheel corner and braking.

Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.