阅读说明：本技术 一种移动自组网路由决策方法 (Routing decision method for mobile ad hoc network ) 是由 王娟 夏琦 高建彬 胡垚 李铭 杜先龄 于 2021-06-29 设计创作，主要内容包括：本发明公开了一种移动自组网路由决策方法,该方法基于可信机制和Q-Learning动态信任值实现,所述可信机制包括邻居监控、信息共享、评估决策和路由管理等。本发明解决了自组网在恶劣情况下节点自身的不信任程度以及节点自主选择最适合的下一跳节点等问题,实现了节点内部评估信任并剔除不信任节点,也实现了节点根据自身探索网络后的智能选择。同时解决了基于信任机制的移动自组网中激励节点诚实推荐的问题,并在此基础上实现可信路由协议。在移动自组网中引入信任机制评判节点的行为,能够有效地建立节点间的信任关系,从而构建一个稳定、可信的网络环境。(The invention discloses a mobile ad hoc network routing decision method which is realized based on a credible mechanism and a Q-Learning dynamic trust value, wherein the credible mechanism comprises neighbor monitoring, information sharing, evaluation decision, routing management and the like. The invention solves the problems of the distrust degree of the nodes of the ad hoc network under severe conditions, the autonomous selection of the most suitable next hop node and the like, realizes the internal evaluation of trust of the nodes and the elimination of distrust nodes, and also realizes the intelligent selection of the nodes after exploring the network according to the nodes. Meanwhile, the truthful recommendation problem of the incentive nodes in the mobile ad hoc network based on the trust mechanism is solved, and a trusted routing protocol is realized on the basis. A trust mechanism is introduced into the mobile ad hoc network to judge the behavior of the nodes, so that the trust relationship between the nodes can be effectively established, and a stable and trusted network environment is constructed.)

1. A mobile ad hoc network routing decision method is characterized in that the method is realized based on a credible mechanism and a Q-Learning dynamic trust value, the credible mechanism comprises neighbor monitoring, information sharing, evaluation decision and routing management, and the method specifically comprises the following steps:

step 1, node A stores information of neighbor node B

If any node A has a direct interaction with the neighboring node B in the t-th time frame, the structure { A: b, action, trust value, t, recording the interactive action, each node maintains the information of the structure and stores the structure in a direct trust value information table, and the trust value marks whether the interactive action is successful or not; the trust relationship between nodes is established by two methods: firstly, the trust information which is acquired locally by the node and is about the evaluated node is called as a direct trust value; the second is the synthesis of trust information about the evaluated nodes from the third party nodes, which is called indirect trust value; the recommendation trust value belongs to a special form of direct trust value and is used for describing the subjective prediction of the truthfulness of a node to other nodes for providing recommendation information;

step 2. quantification of trust value

Step 2.1, directly calculating a trust value: if the node A successfully interacts with the neighbor node B in the ith time frame, k_iSecond, failure j_iSecond, I1, 2.. I, then the probability of selecting node B in the 1 st time frame is:

wherein, t_iIs the time of the ith time frame, i.e. from the 1 st to the ith time frame, t_ITime of the ith time frame, beta₁，β₂Is two preset time weight factors, and beta is more than or equal to 0₁≤1，0≤β₂≤1；

Substituting p into the entropy function H (p):

the trust value is formulated as

Directly trust the value by the above processWherein, the subject represents the node A, the node represents the neighbor node B, and the action represents A, B node interaction condition;

step 2.2, calculating a recommended trust value: and (3) carrying out deviation test on the trust _ value fed back by the neighbor node B and the direct trust value calculated by the node A: judgment ofWhether the preset deviation threshold is satisfied or not, wherein eta represents the preset deviation threshold;

if the deviation test is successful, i.e.If the result is true, the recommendation of the neighbor node B is considered to be honest and reliable, otherwise, the recommendation is considered to be subjected to false recommendation; finally, writing the recommended action into a recommended trust value information table of the node A by the node A, wherein the recommended trust value information table is initialized to be empty; if the recommendation is honest, and the node A successfully interacts with the neighbor node B in the ith time frame by k_iSecond, failure j_iSecond, i 1, 2.. times, M, then the probability of selecting a node B at the mth time frame is:

wherein, t_iTime of ith time frame, t_MTime of Mth time frame, α₁，α₂Two preset time weight factors are included, and alpha is more than or equal to 0₁≤1，0≤α₂≤1；

Bringing p' into

Then a trust value R is recommended_{subject：recommender}Is composed of

Wherein, the subject represents the node A, and the receiver represents the neighbor node B;

step 2.3, calculating an indirect trust value: computing indirect trust values using multi-way recommendations, i.e. recommendation information obtained at multiple neighboring nodesIndirect trust value from node A to node CExpressed as:

wherein R is_A：BRepresenting a recommended trust value, R, from node A to node B_A：DRepresenting a recommended trust value for node a to node D,representing a direct trust value from node B to node C,represents a direct trust value from node D to node C;

and 2.4, calculating the overall trust value: the direct trust value and the indirect trust value are weighted and summed to form the total trust valueWherein α and β are preset weighted values, and α + β is 1;

and 3, carrying out neighbor monitoring when the node A forwards the data packet, judging whether the monitoring is successful or not, updating and maintaining a monitoring cache table peekBuffer, wherein the specific flow of neighbor monitoring is as follows: when a node A sends a data packet to a next hop node B, the node A firstly writes the data packet into a cache peekBuffer, sets a timer, then forwards the data packet to the node B, and simultaneously starts to monitor the node B, if the node A monitors that the node B successfully forwards the data packet, or determines that the node B does not successfully forward the data packet due to the overtime of the timer, the monitored result is transmitted to a trust library, and the data packet is deleted; if the node B does not successfully forward the data packet, the node A needs to initialize information, wherein the trust library comprises a standard cache table buffer, a monitoring cache table peekBuffer, a recommendation request cache waitRecommand buffer and a recommendation information cache table recomBuffer;

and 4, information sharing: after the node A successfully monitors in the step 3, whether the evaluation node B is credible is determined according to a locally stored direct trust value information table, when the node A cannot judge whether a certain node B is credible according to the direct trust value information table, firstly, a local recommended information cache table recomBuffer cached in the node A needs to be checked, for each piece of recommended information related to the node B to be evaluated in the recomBuffer, the node A needs to calculate a recommended trust value, if the calculated recommended trust value is higher than a preset threshold value lambda, the node A is used, otherwise, the piece of recommended information is deleted in the recomBuffer; the multiple nodes maintain the same trust library so as to facilitate the follow-up intelligent cooperation of the multiple nodes;

and step 5, node evaluation: evaluating the recommended trust value of each node, namely performing the deviation test of the step 2.2, wherein the nodes which are lower than the deviation threshold eta are classified as untrusted nodes, the untrusted nodes are not considered in subsequent decision making, and the rest nodes are used as trusted nodes;

step 6, routing management decision: according to the established credible routing protocol, a Q-Learning framework is adopted to simulate the node into an agent in multi-agent cooperation and select the next hop of the node routing.

2. The method for making a routing decision in a mobile ad hoc network according to claim 1, wherein the specific process of information sharing in the step 4 comprises:

step 4.1, the node A initializes recommendation request information and sends a recommendation request to the node B;

step 4.2, the intermediate node B processes the recommendation request information and packages the information into a recommendation feedback information packet;

step 4.3, the intermediate node B processes the recommended feedback information packet and sends the recommended feedback information packet to the node A;

and 4.4, the node A which initiates the recommendation request processes the recommendation feedback information packet and synthesizes an indirect trust value.

3. The method for making a routing decision for a mobile ad hoc network according to claim 2, wherein the routing management decision in step 6 specifically comprises:

step 6.1, based on the trusted nodes obtained in step 5, for any trusted node, the link duration t' and the direct trust value of the node in the trusted routing protocol are calculatedAnd a recommended trust value R_{subject：recommender}Three factors define the reward function R (s, a) in Q-Learning as routing metric information, namely: r (s, a) ═ g + [ μ · t' (s, a) + v · Td (s, a) + λ · Tr (s, a)]. Wherein s is a current node s, a is a selection of a next hop of the node, i.e. the recommended neighbor nodes a, g are reward punishment factors, and t '(s, a), Td (s, a) and Tr (s, a) are t'; respectively,R_{subject：recommender}The normalized values of (a) are preset weighting factors of three factors of t' (s, a), Td (s, a) and Tr (s, a), and the normalized values of (μ + ν + λ) satisfy the condition that μ + ν + λ is 1;

step 6.2. Each NetThe network node is used as a sending node to send the local trust library to the neighbor nodes, and the neighbor nodes are used as receiving nodes to calculate and update Q values according to the respective maintained trust libraries and Q-learning reward functions: q (s, a) ═ R (s, a) + γ max_a′(Q (s ', a')), wherein Q (s, a) is the Q value from node s to node a, R (s, a) is the reward value from node s to node a, γ is a preset discount factor, max_a′(Q (s ', a')) is the Q value from node s 'to node a' when Q value from node s 'to node a' is maximum, and node s 'and node a' are both nodes of the next hop of node s and node a;

and 6.3, when the data packet to be forwarded exists in the node A, intelligently selecting a next hop forwarding node B according to the Q value maintained by the node A, wherein the optimal strategy of the node route is defined as that the behavior with the maximum Q value is selected in each state as the next hop of the node route, and the strategy is as follows: Π(s) ═ argmax_aQ(s，a)。

4. The mobile ad hoc network routing decision method of claim 3, wherein the deviation threshold η has a value in a range of 0 ≦ η < 1.

Technical Field

The invention belongs to the technical field of ad hoc networks, and particularly relates to a mobile ad hoc network routing decision method.

Background

The self-organizing network is a network combining mobile communication and computer network, the information exchange of the network adopts the packet exchange mechanism in the computer network, the user terminal is a portable terminal which can move, each user terminal in the self-organizing network has the functions of a router and a host computer, the self-organizing network is temporarily built without depending on the preset infrastructure, the mobile nodes in the network exchange information by utilizing the wireless transceiving equipment of the mobile nodes, and when the mobile nodes are not in the communication range, the multi-hop communication can be realized by the relay of other intermediate nodes.

Ad hoc networks have some characteristics that general communication networks do not have, such as the ad hoc nature of the network, the dynamic nature of the topology, the limitation of the bandwidth, the distributivity of the control, the multi-hop nature, and the limited security, but also face some challenges: the network topology structure of the ad hoc network is complex and is mostly used under the condition that the environment changes sharply and malicious nodes exist, the communication overhead is large, the throughput bandwidth is limited, the node next hop decision is difficult, and the like.

In an environment with complex and changeable situations, such as an ad hoc network, the state space is extremely large, all problems cannot be solved by using a traditional communication method, the behaviors and actions of the next hop of the ad hoc network node cannot be specified under all situations, and even the routing decision of the ad hoc network can be influenced by the change of hardware equipment and the use environment. Therefore, for the situation, considering the variability of the ad hoc network environment and the security of the node, a trusted routing protocol based on recommended trust is introduced, on the basis of the security, the ad hoc network routing decision is analogized to the multi-agent cooperation and decision problem in the complex environment, and Q-Learning in the field of machine Learning is introduced to improve the intelligent capability of the ad hoc network node for next hop decision.

Disclosure of Invention

In order to overcome the defects, the invention provides a mobile ad hoc network routing decision method, which is realized based on a credible mechanism and a Q-Learning dynamic trust value, wherein the credible mechanism comprises neighbor monitoring, information sharing, evaluation decision, routing management and the like, and the method specifically comprises the following steps:

step 1, node A stores information of neighbor node B

If any node A has a direct interaction with the neighboring node B in the t-th time frame, the structure { A: b, action, trust value, t, recording the interactive action, each node maintains the information of the structure and stores the structure in a direct trust value information table, and the trust value marks whether the interactive action is successful or not; the trust relationship between nodes is established by two methods: firstly, the trust information which is acquired locally by the node and is about the evaluated node is called as a direct trust value; the second is the synthesis of trust information about the evaluated nodes from the third party nodes, which is called indirect trust value; the recommendation trust value belongs to a special form of direct trust value and is used for describing the subjective prediction of the truthfulness of a node to other nodes for providing recommendation information;

step 2, quantizing a trust value: calculating a direct trust value, a recommended trust value, an indirect trust value and an overall trust value;

and 3, carrying out neighbor monitoring when the node A forwards the data packet, judging whether the monitoring is successful or not, updating and maintaining a monitoring cache table peekBuffer, wherein the specific flow of neighbor monitoring is as follows: when a node A sends a data packet to a next hop node B, the node A firstly writes the data packet into a cache peekBuffer, sets a timer, then forwards the data packet to the node B, and simultaneously starts to monitor the node B, if the node A monitors that the node B successfully forwards the data packet, or determines that the node B does not successfully forward the data packet due to the overtime of the timer, the monitored result is transmitted to a trust library, and the data packet is deleted; if the node B does not successfully forward the data packet, the node A needs to initialize information, wherein the trust library comprises a standard cache table buffer, a monitoring cache table peekBuffer, a recommendation request cache waitRecommand buffer and a recommendation information cache table recomBuffer;

and 4, information sharing: after the node A successfully monitors in the step 3, whether the evaluation node B is credible is determined according to a locally stored direct trust value information table, when the node A cannot judge whether a certain node B is credible according to the direct trust value information table, firstly, a local recommended information cache table recomBuffer cached in the node A needs to be checked, for each piece of recommended information of the node B to be evaluated in the recomBuffer, the node A needs to calculate a recommended trust value, if the calculated recommended trust value is higher than a preset threshold value, the node A is used, and otherwise, the piece of recommended information is deleted in the recomBuffer; the multiple nodes maintain the same trust library so as to facilitate the follow-up intelligent cooperation of the multiple nodes;

and step 5, node evaluation: evaluating the recommended trust value of each node to find out a trusted node;

step 6, routing management decision: according to the established credible routing protocol, a Q-Learning framework is adopted to simulate the node into an agent in multi-agent cooperation and select the next hop of the node routing.

The invention solves the problems of the distrust degree of the node self under the severe condition of the ad hoc network, the autonomous selection of the most suitable next hop node by the node and the like. The method realizes internal trust evaluation of the nodes and elimination of distrusted nodes, and also realizes intelligent selection of the nodes after exploring the network according to the nodes. The truthful recommendation problem of the incentive nodes in the mobile ad hoc network based on the trust mechanism is solved, and a credible routing protocol is realized on the basis. A trust mechanism is introduced into the mobile ad hoc network to judge the behavior of the nodes, so that the trust relationship between the nodes can be effectively established, and a stable and trusted network environment is constructed.

Drawings

Fig. 1 is a schematic diagram of node one-way recommendation in the method of the present invention.

FIG. 2 is a schematic diagram of node multi-path recommendation in the method of the present invention.

Fig. 3 is a schematic diagram of the trusted routing protocol in the method of the present invention.

Fig. 4 is a flow chart of node neighbor monitoring in the method of the present invention.

Detailed Description

The invention is further described below with reference to the figures and examples.

The invention provides a mobile ad hoc network routing decision method, which is realized based on a credible mechanism and a Q-Learning dynamic trust value, wherein the credible mechanism comprises neighbor monitoring, information sharing, evaluation decision, routing management and the like, and is shown in figure 3. The method specifically comprises the following steps:

step 1, node A stores information of neighbor node B

If any node A has a direct interaction with the neighboring node B in the t-th time frame, the structure { A: b, action, trust value, t, recording the interactive action, each node maintains the information of the structure, and stores the structure in a direct trust value information table, and trust value marks whether the interactive action is successful or not. The trust relationship between nodes is established by two methods: firstly, the trust information which is acquired locally by the node and is about the evaluated node is called as a direct trust value; the other is the composition of trust information about the evaluated nodes from the third-party nodes, which is called indirect trust value. The recommendation trust value belongs to a special form of direct trust value and is used for describing the subjective prediction of the integrity of other nodes provided by a node. The symbolic description of the trust value is shown in table 1, and the quantized trust value is stored specifically.

TABLE 1 legends

Note: not listing symbols and repeating symbols as appropriate for the occurrence

Step 2. quantification of trust value

And 2.1, directly calculating the trust value. If node a successfully interacts with neighboring node B within the ith time frame (I ═ 1, 2.., I.), k_iSecond, failure j_iSecond, beta₁，β₂For two preset time weighting factors, the probability of selecting the node B in the ith time frame is:

wherein, t_iIs the time of the ith time frame (i.e. the time from the 1 st time frame to the ith time frame), t_IIs the time of the I time frame, and is more than or equal to 0 and less than or equal to beta₁≤1，0≤β₂≤1。

Substituting p into the entropy function H (p):

the trust value is formulated as

Directly trust the value by the above processWherein, the subject represents the node A, the node represents the neighbor node B, and the action represents A, B the interaction condition of the node.

And 2.2, calculating a recommended trust value. And (3) carrying out deviation test on the trust _ value fed back by the neighbor node B and the direct trust value calculated by the node A: judgment ofAnd if the preset deviation threshold is not met, wherein eta represents the preset deviation threshold, and the numeric area is more than or equal to 0 and less than 1.

If the deviation test is successful, i.e.If the result is true, the recommendation of the neighbor node B is considered to be honest and reliable, otherwise, the recommendation is considered to be carried out falsely. And finally, writing the recommended action into a recommended trust value information table of the node A by the node A (wherein the recommended trust value information table is initialized to be empty). According to the value of trust _ value, if the interaction action is successful, the recommendation is honest. If the recommendation is honest, and node a successfully interacts with the neighboring node B within the ith time frame (i 1, 2.. so, M) (to distinguish the two calculation results of the direct trust value and the recommended trust value, i 1, 2.. so, M)_iSecond, failure j_iSub, α₁，α₂For two preset time weighting factors, a person is likely to select the node B in the mth time frame as follows:

wherein, t_iTime of ith time frame, t_MIs the time of the Mth time frame, and is more than or equal to 0 and less than or equal to alpha₁≤1，0≤α₂≤1。

Bringing p' into

Then a trust value R is recommended_{subject：recommender}Is composed of

Wherein, subiect represents node A, and recemmender represents neighbor node B.

And 2.3, calculating an indirect trust value. The indirect trust value calculation is divided into a single-path recommendation type and a multi-path recommendation type, as shown in fig. 1 and fig. 2, respectively, a node can obtain recommendation information from one or more neighbor nodes, the former is called single-path recommendation, and the latter is called single-path recommendationReferred to as multi-way recommendation. The invention considers the complexity of the self-organizing network, adopts multi-path recommendation, namely, calculates indirect trust times through recommendation information obtained by a plurality of neighbor nodesIndirect trust value from node A to node CExpressed as:

wherein R is_A：BRepresenting a recommended trust value, R, from node A to node B_A：DRepresenting a recommended trust value for node a to node D,representing a direct trust value from node B to node C,representing a direct trust value from node D to node C. Wherein A, B, C, D are all node symbols in fig. 2.

And 2.4, calculating the overall trust value. And the direct trust value and the indirect trust value are weighted and summed to form an overall trust value:wherein, alpha and beta are respectively preset manuallyAndwherein, the subject is node a, the node is neighbor node B, the action represents the interactive behavior, and α + β is 1.

And 3, carrying out neighbor monitoring when the node A forwards the data packet, judging whether the monitoring is successful or not, and updating and maintaining the monitoring cache table. As shown in fig. 4, when the node a sends a data packet to the next-hop node B, the node a first writes the data packet into the cache peekBuffer, sets a timer, then forwards the data packet to the node B, and starts monitoring the node B. If the node A monitors that the node B successfully forwards the data packet or determines that the node B does not successfully forward the data packet due to the overtime of the timer, the monitored result is transmitted to the trust library, and the data packet is deleted. If node B fails to successfully forward the packet, node a needs initialization information. The protocol data structure involved therein is shown in table 2.

Table 2 data structure interpretation

Structure of the product	Explanation of the invention
		buffer	Standard buffer memory table
peekBuffer	Monitoring buffer table
		waitRecomBuffer	Recommendation request caching
recomBuffer	Recommended information cache table

And 4, sharing information. After the node a successfully monitors in step 3, it determines whether the evaluation node B is trusted according to the locally stored direct trust value information table, and when the node a cannot determine whether a certain node B is trusted according to the direct trust value information table, first, it needs to check its recompbuffer (the recommended trust value information table in step 2.2 may exist in a link or a cloud, and the recompbuffer refers to an information table cached locally in the node). The multiple nodes maintain the same trust library (the trust library comprises buffer, peekBuffer, waitRecommand buffer, and the like) so as to facilitate the subsequent multi-node intelligent cooperation, and the specific process of information sharing is as follows:

step 4.1, the node A initializes recommendation request information and sends a recommendation request to the node B;

step 4.2, the intermediate node B processes the recommendation request information and packages the information into a recommendation feedback information packet;

step 4.3, the intermediate node B processes the recommended feedback information packet and sends the recommended feedback information packet to the node A;

and 4.4, the node A which initiates the recommendation request processes the recommendation feedback information packet and synthesizes an indirect trust value.

And 5, evaluating the nodes. And (3) evaluating the recommended trust value of each node, namely performing the deviation test of the step 2.2, and listing the node which is lower than the threshold eta as an untrusted node. Subsequent decisions do not take these non-trusted nodes into account, with the remaining nodes being trusted nodes.

And 6, making a routing management decision. According to the established trusted routing protocol, a Q-Learning framework is adopted to simulate the nodes into the intelligent agents in multi-agent cooperation.

Step 6.1, based on the trusted node obtained in the step 5, for any trusted node, the link duration t' and the direct trust value of the node in the trusted routing protocol are calculatedAnd recommending trust valuesThree factors define the reward function R (s, a) in Q-Learning as routing metric information, namely: r (s, a) ═ g + [ μ · t' (s, a) + v · Td (s, a) + λ · Tr (s, a)]. Wherein s is a current node s, a is a selection of a next hop of the node, i.e. the recommended neighbor nodes a, g are reward punishment factors, and t '(s, a), Td (s, a) and Tr (s, a) are t'; respectively,R_{subject：recommender}The normalized values of (a) are weight factors preset by three factors of t' (s, a), Td (s, a) and Tr (s, a), and satisfy the condition that μ + v + λ is 1.

Step 6.2, each network node is used as a sending node to send the respective local trust library to the neighbor nodes, and the neighbor nodes are used as receiving nodes to calculate and update Q values according to the respective maintained trust libraries and Q-learning reward functions: q (s, a) ═ R (s, a) + γ max_a′(Q (s ', a')), wherein Q (s, a) is the Q value from node s to node a, R (s, a) is the reward value from node s to node a, γ is a preset discount factor, max_a′(Q (s ', a')) is the Q value from node s 'to node a' when Q value from node s 'to node a' is the maximum value, and node s 'and node a' are both nodes of the next hop of node s and node a.

And 6.3, when the data packet to be forwarded exists in the node A, intelligently selecting a next hop forwarding node B according to the Q value maintained by the node A, wherein the optimal strategy of the node route is defined as that the behavior with the maximum Q value is selected in each state as the next hop of the node route, and the strategy is as follows: Π(s) ═ argmax_aQ(s，a)。

The embodiments described above are only a part of the embodiments of the present invention, and not all of them. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.