阅读说明：本技术 一种分布式图数据库数据处理方法、介质和电子设备 (Distributed graph database data processing method, medium and electronic equipment ) 是由 杨娟 杨再飞 翟士丹 杨珂 沈游人 武永卫 陈康 姜进磊 吴昊 于 2021-09-29 设计创作，主要内容包括：本申请实施例公开了一种分布式图数据库数据处理方法、介质和电子设备。所述方法包括：获取数据处理请求,并根据数据处理请求类型和处理节点角色之间的关联关系,确定处理所述数据处理请求的节点角色作为目标角色；获取当前角色信息,若当前角色与所述目标角色相匹配,则根据预设数据处理规则对所述数据处理请求进行处理。执行本申请技术方案,能够提高分布式图数据库的数据处理效率,充分发挥分布式图数据库的数据处理性能。(The embodiment of the application discloses a distributed graph database data processing method, a distributed graph database data processing medium and electronic equipment. The method comprises the following steps: acquiring a data processing request, and determining a node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of a processing node; and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role. By executing the technical scheme, the data processing efficiency of the distributed graph database can be improved, and the data processing performance of the distributed graph database can be fully exerted.)

1. A method of distributed graph database data processing, the method comprising:

acquiring a data processing request, and determining a node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of a processing node;

and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role.

2. The method of claim 1, wherein the node roles comprise: at least one of a head node, a tail node, and a learner node; the node roles are predetermined according to performance attributes of the distributed nodes.

3. The method of claim 2, wherein the current role is a head node; correspondingly, the processing the data processing request according to the preset data processing rule includes:

responding to the received data writing request, and acquiring role information and communication addresses of all current distributed nodes;

determining a target multicast group address according to the current role and the communication address of each distributed node;

forwarding the data write request to a node associated with the target multicast group address so that the node associated with the target multicast group address writes the data write request into a log.

4. The method of claim 3, wherein the current role is a tail node, and correspondingly, processing the data processing request according to a preset data processing rule comprises:

counting the number of the log success messages in response to receiving the log write success message; wherein the log success message is sent by a node associated with the target multicast group address;

and generating feedback information according to the number of the log write-in success messages acquired within a set time length and the number of the nodes associated with the target multicast group address, and broadcasting the feedback information to the nodes associated with the target multicast group address.

5. The method of claim 4, wherein the current role is a head node; correspondingly, the processing the data processing request according to the preset data processing rule includes:

in response to receiving node role update information, updating the target multicast group address according to the node role update information;

and forwarding the data write-in request to a node associated with the updated multicast group address according to the updated multicast group address.

6. The method of claim 4, wherein the current role is a learner node; correspondingly, the processing the data processing request according to the preset data processing rule includes:

and responding to the received writing success message, and storing the data to be written in the data writing request in a column storage mode.

7. The method of claim 2, wherein the current role is a tail node; correspondingly, the processing the data processing request according to the preset data processing rule includes:

and responding to the received data reading request, reading the data to be read in the local graph database according to the data reading request, and feeding back the data to be read.

8. The method of claim 2, wherein the current role is a learner node; correspondingly, the processing the data processing request according to the preset data processing rule includes:

and responding to the received data analysis request, reading the data to be analyzed in the local graph database according to the data analysis request, and feeding back the data to be analyzed.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for distributed graph database data processing according to any one of claims 1-8.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the distributed graph database data processing method according to any of claims 1-8 when executing the computer program.

Technical Field

The embodiment of the application relates to the technical field of computer application, in particular to a distributed graph database data processing method, a distributed graph database data processing medium and electronic equipment.

Background

In recent years, with rapid development of industries such as social contact, finance, internet of things and the like, distributed graph databases appear like spring shoots after rain. When data is written into a distributed database, the reliability and availability of the data are generally ensured by means of data backup. The backed up data is called a copy, and it is important to ensure the consistency of multiple copies.

Most distributed graph databases currently use a voting-based data replication protocol to ensure consistency of replicas, for example, the Nebula distributed graph database uses the Raft protocol. However, all data reading and writing in the Raft protocol are processed by the leader node, so for reading and writing of hotspot data, the leader role in the Raft protocol becomes a performance bottleneck.

Disclosure of Invention

The embodiment of the application provides a distributed graph database data processing method, a medium and electronic equipment, which can achieve the purposes of improving the data processing efficiency of a distributed graph database and fully exerting the data processing performance of the distributed graph database.

In a first aspect, an embodiment of the present application provides a method for processing distributed graph database data, where the method includes: acquiring a data processing request, and determining a node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of a processing node;

and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role.

In a second aspect, an embodiment of the present application provides a distributed graph database data processing apparatus, including:

the target role determining module is used for acquiring the data processing request and determining the node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of the processing node;

and the data processing module is used for acquiring the current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role.

In a third aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a method for processing distributed graph database data according to embodiments of the present application.

In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable by the processor, where the processor executes the computer program to implement a method for processing distributed database data according to an embodiment of the present application.

According to the technical scheme provided by the embodiment of the application, a node role for processing a data processing request is determined as a target role by acquiring the data processing request and according to the incidence relation between the type of the data processing request and the role of a processing node; and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role. According to the method and the device, the roles of the nodes corresponding to the distributed graph database are distinguished, and the roles of the nodes are associated with the types of the data processing requests. Nodes in different roles process different types of data processing requests. The data processing does not depend on the nodes of one role any more, the data processing efficiency of the distributed graph database can be improved, and the data processing performance of the distributed graph database can be fully exerted.

Drawings

FIG. 1 is a flowchart of a distributed graph database data processing method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a distributed graph database data processing apparatus according to a second embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.

Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.

Example one

Fig. 1 is a flowchart of a distributed graph database data processing method according to an embodiment of the present application, which is applicable to processing data using a distributed graph database, for example, writing or reading data to or from the distributed graph database. The method can be executed by the distributed graph database data processing device provided by the embodiment of the application, and the device can be realized by software and/or hardware and can be integrated in the electronic equipment for running the system.

As shown in fig. 1, the distributed graph database data processing method includes:

s110, acquiring the data processing request, and determining the node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of the processing node.

The data processing request comprises an external data processing request sent by an external user and acquired by a distributed node corresponding to the distributed graph database, and the specific external data processing request comprises: at least one of a data write request, a data read request, or a data analysis request. The method may also include that, during the process of processing the external data processing request, the distributed node generates an internal data processing request that needs to be processed by the distributed node, and the example includes: the log writing request and the feedback information are log writing success information and log writing success information. The distributed graph database may be a distributed graph database.

The incidence relation between the data processing request type and the processing node role is predetermined and written into each distributed node by related technicians according to actual service requirements. The association relationship between the data processing request type and the processing node role property is not limited herein, and is determined according to actual situations. Specifically, the distributed graph database data processing in the application adopts a read-write separation design, and the role of a node responsible for processing the data write-in request is different from the role of a node responsible for processing the data write-in request.

In an alternative embodiment, the node roles include: at least one of a head node, a tail node, and a learner node; the node roles are predetermined according to performance attributes of the distributed nodes.

Specifically, the node role is determined by the metadata server according to the performance attributes of each distributed node, such as resource utilization rate and data throughput, optionally, a node with better performance is selected from the distributed nodes as a head node, a tail node and a learner node, and an intermediate node is selected from the rest distributed nodes. The number of the head node, the tail node and the learner node is 1, the number of the intermediate nodes can be determined according to actual business requirements, specifically, the number of the intermediate nodes can be determined according to the number of the distributed graph database data backups, and the number is not limited herein. Illustratively, if the reliability and the availability of data are ensured by the distributed graph database in a three-copy mode, a head node and a tail node are removed, and only one intermediate node needs to be arranged; if the reliability and the availability of the data are ensured by the distributed graph database in a four-copy mode, two intermediate nodes are required to be arranged after the head node and the tail node are removed.

Optionally, in the case of processing an external data processing request, the head node is responsible for processing a data write request; the tail node is responsible for processing a data reading request; the learner node is responsible for processing data analysis requests. That is, the target role corresponding to the data write request is the head node; the target role corresponding to the data reading request is a tail node; the target role corresponding to the data analysis request is the learner node.

The distributed nodes belonging to the distributed graph database system can receive the data processing request, the distributed nodes obtaining the data processing request can determine the type of the data request, and the node role processing the data processing request is determined as a target role according to the preset incidence relation between the type of the data processing request and the role of the processing node.

And S120, acquiring current role information, and if the current role is matched with the target role, processing the data processing request according to a preset data processing rule.

Abnormal conditions such as downtime and the like of distributed nodes in a distributed graph database system can occur, and the roles of the distributed nodes can be changed due to the change of the performance of the distributed nodes. The possible network partitioning situation may result in the node roles locally stored by the distributed nodes not being updated in time. Therefore, after receiving the data processing request, the distributed node needs to obtain the current role information from the metadata server, and update the original role information according to the current role information. Wherein the current role information includes current roles of all distributed nodes in the distributed graph database system. That is, the current role information includes both the current role of the current node and the current roles of other distributed nodes.

And after the distributed node receiving the data processing request to be processed acquires the current role information, matching the current role of the distributed node with the target role, and if the current role is consistent with the target role, processing the data processing request according to a preset data processing rule. Wherein the preset data processing rule is related to the type of the data processing request. The preset data processing rules corresponding to the different types of data processing requests are different. The specific content of the preset data processing rule is not limited herein, and is determined according to the actual situation, for example, the preset data processing rule may be forwarding of a data processing request, reading or writing of data, and the like.

And if the current role of the distributed node receiving the data processing request is not matched with the target role. The node receiving the data processing request forwards the received data processing request to the current role as the target role distributed node according to the acquired current information. And processing the data processing request by the node with the current role as the target role according to a preset data processing rule.

Specifically, in an optional embodiment, the current role is a tail node; the target role is matched with the current role, and correspondingly, the data processing request is processed according to a preset data processing rule, wherein the processing method comprises the following steps: and responding to the received data reading request, reading the data to be read in the local graph database according to the data reading request, and feeding back the data to be read.

And the tail node responds to the received data reading request, reads the data to be read from the local graph database maintained by the tail node according to the data reading request, feeds back the data to be read, and specifically feeds back the read data to be read to the user through a calculation layer in the distributed graph database system. The data read request is a transaction-type data read request, such as a data query request generated by an external user.

In an alternative embodiment, the current role is a learner node; the target role is matched with the current role, and correspondingly, the data processing request is processed according to a preset data processing rule, wherein the processing method comprises the following steps: and responding to the received data analysis request, reading the data to be analyzed in the local graph database according to the data analysis request, and feeding back the data to be analyzed.

And the learner node responds to the received data analysis request, reads the data to be analyzed from the local map database maintained by the learner node according to the data analysis request, and feeds back the data to be analyzed. The read data to be analyzed can be fed back to the user through a computing layer in the distributed graph database system. The data analysis request is an analytic data reading request, such as a community mining request generated by an external user.

It is noted that, the computation layer arranged in the distributed graph database system is defined with core logic for processing data processing requests, and the underlying graph database corresponding to the distributed nodes is responsible for providing basic data support. According to the method and the device, the calculation layer in the distributed graph database system is not described in an expansion mode, and the distributed data, namely the data processing process of the distributed nodes, is focused. The computation layer provides data support by the tail node in the process of processing the data reading request. The computing layer provides data support by the learner node in processing the data analysis request.

Therefore, in the process of processing the data reading request and the data analysis request, the node roles are distinguished, and the roles are determined by different data storage modes of the tail node and the learner node. This allows the distributed graph database to better support HTAP (Hybrid Transaction and Analytical Processing) services. In an alternative embodiment, the learner node stores data in a column store and the tail node stores data in a row store. The special data storage mode of the learner node enables the data to be analyzed to be read from the learner node more effectively when the data analysis is carried out on the computing layer. In general, when storing map data, a distributed map database stores data in a row. The line storage mode, the point edge and the respective attribute value are stored together. The data writing efficiency of the graph data is high, the data integrity can be guaranteed, and the requirement for fast query of the graph data can be met. But the way the rows are stored there is data redundancy. When data is stored in a line memory, a large amount of unnecessary attribute data is read out during graph analysis. In contrast, the storage mode of column storage has higher data reading efficiency, no data redundancy, the valid data can be compactly stored together, the same attribute values are stored together, and the connection of the time and the edge is stored together by using a more compact csr (compressed spare row), so that the graph analysis performance can be greatly improved.

In an alternative embodiment, the current role is a head node; the target role is matched with the current role, and correspondingly, the data processing request is processed according to a preset data processing rule, wherein the processing method comprises the following steps: responding to the received data writing request, and acquiring role information and communication addresses of all current distributed nodes; determining a target multicast group address according to the current role and the communication address of each distributed node; forwarding the data write request to a node associated with the target multicast group address so that the node associated with the target multicast group address writes the data write request into a log.

In the process of processing data write requests by a distributed graph database, data consistency and availability are generally ensured by means of data backup. That is, the same data will store multiple copies in the distributed graph database. In order to ensure that the data availability is not affected by the number of copies, the embodiment of the application uses a multicast copy protocol to copy data in real time to obtain data copies, and a head node is responsible for broadcasting a data write request to a distributed node which needs to store data and storing the data to obtain the copies.

In order to avoid the problem that the role of the node is not updated timely due to network partitioning, after the head node receives a data write request, the head node needs to acquire the latest role information and communication addresses of each distributed node from the metadata server again, and determine a target multicast group address according to the current role and communication addresses of the distributed nodes. The target multicast group address is a communication address of the distributed node needing to store the copy. Utensil for cleaning buttock

And in the body, the head node adds the communication addresses with the current roles of the tail node and the intermediate node into the target multicast group address. The destination multicast group address also includes the communication address of the head node itself. The head node forwards the data write request to the node associated with the target multicast group address, and the node associated with the target multicast group address writes the data write request into the log. The data writing request comprises data to be written.

After the data write request is written into the log by the node associated with the target multicast group address, a log write success message is generated, and the log write success message is sent to the tail node.

In order to further improve the data processing efficiency of the distributed graph database and fully exert the performance of the distributed graph database system, internal processing requests generated in the process of processing external data processing requests such as data writing requests by the distributed graph database system are also processed by different role nodes.

In an optional embodiment, the current role is a tail node, the target role is matched with the current role, and correspondingly, the processing of the data processing request according to a preset data processing rule includes: counting the number of the log success messages in response to receiving the log write success message; wherein the log success message is sent by a node associated with the target multicast group address; and generating feedback information according to the number of the log write-in success messages acquired within a set time length and the number of the nodes associated with the target multicast group address, and broadcasting the feedback information to the nodes associated with the target multicast group address.

The tail node counts the number of successful log messages in response to receiving the successful log writing message; the log success message is generated and sent to the tail node by the head node and the intermediate node in the target multicast group address after the log writing is successful. The tail node itself generates a log write success message after the data write request is successfully written into the log. In the process of counting the number of successful log messages by the tail node, the successful log write messages generated by the tail node are counted. And the tail node generates feedback information according to the number of the log write-in success messages acquired within the set time length and the number of the nodes associated with the target multicast group address, and broadcasts the feedback information to the nodes associated with the target multicast group address.

The set time is determined by the related technical personnel according to the actual service requirement, and the specific value is not limited herein. If the number of the log write-in success messages acquired by the tail node within the set time is consistent with the number of the nodes associated with the target multicast group address, it is indicated that all the distributed nodes needing to store the copies have received the data write-in request and have written the data write-in request into the log. At this time, the feedback information generated by the tail node is a write success message; optionally, the target multicast group address may further include an address with a role of a learner node, but the learner node does not write the data write request into the log after receiving the data write request forwarded by the head node, and does not feed back the log to the head node. And only after receiving the writing success message broadcast by the tail node, the learner node stores the data to be written in the data writing request with the head node and the middle node. Meanwhile, the tail node stores the data to be written in the data writing request. Optionally, the head node and the tail node are stored in a row storage manner, and the learner node is stored in a column storage manner. It should be noted that, in the case that the target multicast group address includes the learner node, since the learner does not write the data writing request into the log and does not feed back the log condition to the head node, the number of nodes associated with the target multicast group address should be subtracted by the number of learner nodes.

To ensure data consistency and availability, the tail node generates and broadcasts a write success message only after all nodes associated with the target multicast group address have successfully logged. And each node associated with the target multicast group address stores the data to be written in the data writing request only after receiving the writing success message broadcast by the tail node.

If the number of the log write-in success messages acquired by the tail node within the set time is not consistent with the number of the nodes associated with the target multicast group address, the tail node does not generate feedback information. This indicates that there is an abnormal situation such as a down of the distributed node or a network interruption, so that the distributed node that needs to store the copy fails to successfully write the data write request into the log.

In order to ensure that the distributed database system can operate reliably and stably, the metadata server monitors the operation state of each distributed node in the distributed database system in the operation process of the distributed database system. When the metadata server monitors that the running state of the distributed nodes is abnormal, the metadata server initiates view change (view change) to update the state of the whole cluster. Specifically, the metadata server may update the entire cluster state by replacing the data storage node and reassigning the role of the node. For example, the metadata server may select a distributed node in an idle state and with better performance to replace a distributed node with an abnormal operating state according to the operating state and the performance attribute of the distributed node, so as to update the node role. The metadata server generates node role update information according to the node update condition, and broadcasts the node role update information in the distributed database system, so that the distributed nodes update the current roles of the distributed nodes and the current role information of other locally stored distributed nodes according to the node role update information broadcast by the metadata server. After view change (view change) initiated by the metadata server is finished, the distributed graph database system can recover the data processing flow under the role configuration of the new node, and after the data processing flow is recovered, the distributed graph database system can continue to provide data processing service for external users. In an alternative embodiment, the current role is a head node, and the target role is matched with the current role; correspondingly, the processing the data processing request according to the preset data processing rule includes: in response to receiving node role update information, updating the target multicast group address according to the node role update information; and forwarding the data write-in request to a node associated with the updated multicast group address according to the updated multicast group address.

The node role update information is generated by a metadata server, all distributed nodes in a specific distributed graph database system can monitor the update information broadcast by the metadata server through heartbeat, once the metadata server updates the cluster state, all distributed nodes can monitor the node role update information, and the current role information of each distributed node is updated according to the node role update information. Of course, a new header node may also be generated during the update of the cluster state by the metadata server.

And the head node responds to the received role updating information and updates the target multicast group address according to the node role updating information. The head node is the head node designated by the metadata server in the process of updating the cluster state. The head node updates the target multicast group address according to the node role update information, specifically, the head node can reconstruct a new target multicast group address according to the node role update information, and directly update the old target multicast group address before the new target multicast group address. And removing the nodes with abnormal operation states from the target multicast group according to the node role update information, and adding the substitute nodes into the target multicast group to obtain an updated target multicast group address.

And the head node forwards the data write-in request to the nodes associated with the updated multicast group address according to the updated multicast group address, so that each node associated with the updated multicast group address writes the data write-in request into the log.

According to the technical scheme provided by the embodiment of the application, a node role for processing a data processing request is determined as a target role by acquiring the data processing request and according to the incidence relation between the type of the data processing request and the role of a processing node; and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role. According to the method and the device, the roles of the nodes corresponding to the distributed graph database are distinguished, and the roles of the nodes are associated with the types of the data processing requests. Nodes in different roles process different types of data processing requests. The data processing does not depend on the nodes of one role any more, the data processing efficiency of the distributed graph database can be improved, and the data processing performance of the distributed graph database can be fully exerted.

In a specific embodiment, when a head node in a distributed graph database system receives a data write request, the flow of the distributed graph database to data processing method is as follows: after receiving a data writing request, the head node acquires role information and communication addresses of all current distributed nodes from a metadata server; and adding the communication addresses of the distributed nodes with the roles of the intermediate node, the learner node and the tail node into the target multicast group address, and broadcasting the data write request to the distributed nodes associated with the target multicast group address. The data writing request comprises data to be written, and a network communication mode of grpc communication is adopted among the distributed nodes. After the intermediate node and the tail node receive the data write request sent by the head node, the data write request is written into the log. And generating a log write success message after the log write success, sending the message to the tail node, counting the number of the messages of the log write success message by the tail node, and generating broadcast feedback information according to the number of the messages and the number of nodes of the message sending node. The process that the head node and the intermediate node feed back the log writing situation to the tail node is the process of determining the data consistency. It should be noted that, the learner node does not participate in the determination process of data consistency, after receiving the data processing request forwarded by the head node, the learner node does not need to write the data processing request into the log, and the learner node does not need to feed back the log writing situation to the tail node.

And only under the condition that the writing success information is broadcast by the tail node, the learner node stores the data to be written in the data writing request with the head node, the tail node and the middle node. Specifically, the learner node caches data to be written in a memory, and writes the data into the memory by a batch method when the cache reaches a certain size. At this time, it can be determined that the data processing request is processed and the data to be written is successfully backed up.

The distributed graph database system is also provided with metadata service for monitoring the operation state of each distributed node in the distributed graph database system in the operation process of the distributed graph database system. When the metadata server monitors that the running state of the distributed nodes is abnormal, the metadata server initiates view change (view change) to update the state of the whole cluster. Specifically, the metadata server may update the entire cluster state by replacing the data storage node and reassigning the node role, and the metadata server may further generate node role update information according to the node update condition and broadcast the node role update information in the distributed database system, so that the distributed nodes update their own current roles and locally stored current role information of other distributed nodes according to the node role update information broadcast by the metadata server.

And the distributed node with the current role as the head node updates the target multicast group address according to the node role update information, forwards the data write-in request to the node associated with the updated multicast group address according to the updated multicast group address, and executes the process again until the backup of the data to be written is successful and feeds back a message of successful write-in to the user.

Currently, the data replication protocol for distributed graph databases mostly uses the Raft protocol because the Raft protocol is easy to understand and implement. However, the Raft protocol works through a Quorum mechanism, namely a voting mechanism, and more than half of nodes are required to participate to work. Thus in the case of two copies, Raft requires the collection of votes for both copies to make a copy of each log. At this time, if one node is down, the Raft protocol cannot continue to work under the condition that one copy is left.

The method and the device utilize the multicast replication protocol to replicate the data in real time to realize data backup, so that the strong consistency and high availability of the data are ensured. The data write request is processed by the head node, and the head node is responsible for broadcasting the data write request to other nodes. When the data is stored completely, the data writing request is processed completely, the multicast replication protocol does not work through a voting mechanism, but works in a multicast full replication mode, and therefore the multicast replication protocol can continue to work even if only one copy remains. The distributed graph database can therefore be used to provide services to the outside even in the case of one copy (in the case of a downtime in one of the two copies).

Different from the fact that all data reading and writing in the Raft protocol are carried out through leader nodes, the read-write separation technology is adopted, the data reading request is processed by tail nodes, and the data analysis request is processed by the tail nodes, so that the throughput of hot data reading and writing can be greatly improved, the data processing performance of a distributed graph database is fully exerted, and higher and more stable performance is provided under the condition of reading intensive loads.

The roles of all nodes in the multicast replication protocol are specified by the metadata server in the present application. When the state of the nodes changes, if the node is down, the roles of all the nodes are also specified by the metadata server in the recovery process, rather than being determined by an election protocol. The node state is directly managed by the metadata server instead of being managed by a replication protocol, so that the complexity and the development difficulty of the system are greatly reduced.

Example two

Fig. 2 is a schematic structural diagram of a distributed graph database data processing apparatus according to a second embodiment of the present application, which is applicable to processing data using a distributed graph database, for example, writing or reading data to or from the distributed graph database. The apparatus can be implemented by software and/or hardware, and can be integrated in an electronic device running the system.

As shown in fig. 2, the apparatus may include: a target role determination module 310 and a data processing module 320.

A target role determination module 310, configured to obtain a data processing request, and determine, according to an association relationship between a type of the data processing request and a role of a processing node, the role of the processing node that processes the data processing request as a target role;

and the data processing module 320 is configured to obtain current role information, and if the current role matches the target role, process the data processing request according to a preset data processing rule.

According to the technical scheme provided by the embodiment of the application, a node role for processing a data processing request is determined as a target role by acquiring the data processing request and according to the incidence relation between the type of the data processing request and the role of a processing node; and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role. According to the method and the device, the roles of the nodes corresponding to the distributed graph database are distinguished, and the roles of the nodes are associated with the types of the data processing requests. Nodes in different roles process different types of data processing requests. The data processing does not depend on the nodes of one role any more, the data processing efficiency of the distributed graph database can be improved, and the data processing performance of the distributed graph database can be fully exerted.

Optionally, the node roles include: at least one of a head node, a tail node, and a learner node; the node roles are predetermined according to performance attributes of the distributed nodes.

Optionally, the current role is a head node; accordingly, the data processing module 320 includes: the information acquisition submodule is used for responding to the received data writing request and acquiring the role information and the communication address of each current distributed node; a target multicast group address determining submodule, configured to determine a target multicast group address according to the current role and the communication address of each distributed node; and the data write request forwarding submodule is used for forwarding the data write request to the node associated with the target multicast group address so that the node associated with the target multicast group address writes the data write request into a log.

Optionally, the current role is a tail node, and correspondingly, the data processing module 320 includes: the message quantity counting submodule is used for responding to the received log writing success message and counting the quantity of the log writing success message; wherein the log success message is sent by a node associated with the target multicast group address; and the feedback information generation submodule is used for generating feedback information according to the number of the log write-in success messages acquired within the set duration and the number of the nodes associated with the target multicast group address, and broadcasting the feedback information to the nodes associated with the target multicast group address.

Optionally, the current role is a head node; accordingly, the data processing module 320 includes: the node role updating information polling submodule is used for responding to the received node role updating information; and the data write-in request retransmission submodule is used for updating the target multicast group address according to the node role updating information and retransmitting the data write-in request to the node associated with the updated multicast group address according to the updated multicast group address.

Optionally, wherein the current role is a learner node; accordingly, the data processing module 320 includes: and the to-be-written data storage module is specifically used for storing the to-be-written data in the data writing request in a column storage mode in response to receiving the writing success message.

Optionally, the current role is a tail node; accordingly, the data processing module 320 includes: and the data reading request processing submodule is specifically used for responding to the received data reading request, reading the data to be read in the local graph database according to the data reading request, and feeding back the data to be read.

Optionally, wherein the current role is a learner node; accordingly, the data processing module 320 includes: and the data analysis request processing submodule is specifically used for responding to a received data analysis request, reading data to be analyzed in the local graph database according to the data analysis request, and feeding back the data to be analyzed.

The distributed graph database data processing device provided by the embodiment of the invention can execute the distributed graph database data processing method provided by any embodiment of the invention, and has corresponding performance modules and beneficial effects for executing the distributed graph database data processing method.

EXAMPLE III

A third embodiment of the present application further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for distributed graph database data processing, the method comprising:

acquiring a data processing request, and determining a node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of a processing node;

and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role.

Storage media refers to any of various types of memory electronics or storage electronics. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the computer system in which the program is executed, or may be located in a different second computer system connected to the computer system through a network (such as the internet). The second computer system may provide the program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different unknowns (e.g., in different computer systems connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.

Of course, the storage medium provided in the embodiments of the present application contains computer-executable instructions, and the computer-executable instructions are not limited to the distributed graph database data processing method described above, and may also perform related operations in the distributed graph database data processing method provided in any embodiments of the present application.

Example four

The fourth embodiment of the present application provides an electronic device, where the distributed graph database data processing apparatus provided in the embodiment of the present application may be integrated into the electronic device, and the electronic device may be configured in a system, or may be a device that performs part or all of the functions in the system. Fig. 3 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application. As shown in fig. 3, the present embodiment provides an electronic device 400, which includes: one or more processors 420; a storage device 410, configured to store one or more programs, when the one or more programs are executed by the one or more processors 420, so that the one or more processors 420 implement the distributed graph database data processing method provided in this application, the method including:

acquiring a data processing request, and determining a node role for processing the data processing request as a target role according to the incidence relation between the type of the data processing request and the role of a processing node;

and acquiring current role information, and processing the data processing request according to a preset data processing rule if the current role is matched with the target role.

Of course, those skilled in the art will appreciate that the processor 420 may also implement the solution of the distributed graph database data processing method provided in any of the embodiments of the present application.

The electronic device 400 shown in fig. 3 is only an example and should not bring any limitations to the performance and the scope of use of the embodiments of the present application.

As shown in fig. 3, the electronic device 400 includes a processor 420, a storage device 410, an input device 430, and an output device 440; the number of the processors 420 in the electronic device may be one or more, and one processor 420 is taken as an example in fig. 3; the processor 420, the storage device 410, the input device 430, and the output device 440 in the electronic apparatus may be connected by a bus or other means, and are exemplified by a bus 450 in fig. 3.

The storage device 410 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and module units, such as program instructions corresponding to the distributed graph database data processing method in the embodiment of the present application.

The storage device 410 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for performance; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage 410 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, storage 410 may further include memory located remotely from processor 420, which may be connected via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input means 430 may be used to receive input numbers, character information, or voice information, and to generate key signal inputs related to user settings and performance control of the electronic device. The output device 440 may include a display screen, speakers, or other electronic equipment.

The distributed graph database data processing device, the medium and the electronic device provided in the above embodiments may execute the distributed graph database data processing method provided in any embodiment of the present application, and have corresponding performance modules and beneficial effects for executing the method. Technical details that are not described in detail in the above embodiments may be referred to a distributed graph database data processing method provided in any of the embodiments of the present application.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.