阅读说明：本技术 批量消息队列的校验方法、电子设备和介质 (Batch message queue checking method, electronic device and medium ) 是由 邵佳文 于 2021-08-16 设计创作，主要内容包括：本发明公开了一种批量消息队列的校验方法、电子设备和介质,其特征在于,其中,校验方法包括以下步骤：步骤S1：上游系统将消息队列发送给下游系统,所述消息队列带有一版本号,步骤S2：下游系统接受所述消息队列,步骤S3：下游系统校验所述消息队列。本发明避免了上游系统逐条发送下游系统逐条消费,也避免在同一时间将全部消息队列取出,减小了CPU的使用量,减小了系统的风险,也降低了系统的复杂程度,有利于系统的代码的开发,增强了可推广性。(The invention discloses a verification method, electronic equipment and a medium for a batch message queue, which are characterized in that the verification method comprises the following steps: step S1: the upstream system sends a message queue to the downstream system, the message queue having a version number, step S2: the downstream system accepts the message queue, step S3: the downstream system checks the message queue. The invention avoids the upstream system sending the downstream system one by one for consumption, and also avoids taking out all message queues at the same time, thereby reducing the use amount of CPU, reducing the risk of the system, reducing the complexity of the system, being beneficial to the development of the code of the system, and enhancing the generalizability.)

1. A method for verifying a batch message queue, the method comprising the steps of:

step S1: the upstream system sends a message queue to the downstream system, wherein the message queue is provided with a version number;

step S2: the downstream system accepts the message queue;

step S3: the downstream system checks the message queue;

the step S1 specifically includes: when the message queue sent by the upstream system is the last message queue in the message queues with the version numbers, the last message queue comprises a last flag bit and a message batch number field;

wherein the step S2 specifically includes: the downstream system extracts the message queue with the version number into a database, identifies and counts the version number, establishes a set corresponding to the version number in the database one by one, and stores the message queue with the version number in the set;

when a downstream system accepts the last message of the message queue, reading the last mark and the message batch number field;

wherein step S3 specifically includes: and the downstream system compares the counting result of the message queue with the version number with the message batch number field.

2. The method for validating a bulk message queue as in claim 1, wherein the database serves a remote dictionary.

3. The method for verifying the batch message queue according to claim 1, wherein the step S2 comprises the steps of:

step S21: judging whether the database comprises a set corresponding to the version number, if so, adding the message queue to the set; and if not, the database creates a set corresponding to the version number, and the message queue is stored in the created set.

4. The method of checking a bulk message queue of claim 1, wherein:

the step S3 includes the steps of:

step S31: when the message batch number field is different from the counting result, the downstream system informs the upstream system of sending a failure message to the message queue; step S32: when the message batch number field is the same as the counting result, the downstream system informs the upstream system that the message queue is successfully sent.

5. The method of verifying a bulk message queue of claim 4, wherein:

the step S3 further includes the steps of:

step S33: and when the upstream system receives a transmission failure message, the upstream system transmits the message queue again.

6. The method of verifying a bulk message queue of claim 5, wherein: the step S33 further includes the downstream system counting the number of times of sending the failure message.

7. The method of verifying a bulk message queue of claim 6, wherein: the step S3 further includes the steps of:

step S34: a threshold value is preset in the upstream system, and the threshold value is used for comparing with the sending failure times;

if the number of transmission failures exceeds the threshold, the upstream system stops transmitting;

and if the sending failure times do not exceed the threshold value, the upstream system resends the message queue once.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the batch message queue checking method of any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of checking a bulk message queue according to any one of claims 1 to 7.

Technical Field

The invention belongs to the technical field of information, and particularly relates to a batch message queue checking method, electronic equipment and a medium.

Background

In the current stage, the mechanism for consuming the message queue is that the upstream system pushes one by one and the downstream system receives one by one, but due to network jitter or host failure, the information transmission may be lost or out of sequence.

When an enterprise solves batch message data at the present stage, the messages are all received and stored in the system uniformly, and then the system extracts the required data from the inside, but the resource method comprises the following steps: when the data volume of the upstream system is too large or the batches of the data are too large, the CPU pressure of the downstream system is too large, even the down condition occurs in a peak period, the uniform taking-out operation cannot achieve the quasi-real-time performance, the uniform data can only be taken out through the pre-appointed time point between the upstream system and the downstream system, and then the batch judgment is carried out.

Disclosure of Invention

The invention provides a batch message processing method, electronic equipment and a medium, aiming at overcoming the defects that in the prior art, the upstream and downstream systems push and consume one by one and all messages are taken out at the same time.

The invention solves the technical problems through the following technical scheme:

a method for verifying a batch message queue comprises the following steps:

step S1: the upstream system sends a message queue to the downstream system, wherein the message queue is provided with a version number;

step S2: the downstream system accepts the message queue;

step S3: the downstream system checks the message queue;

step S1 specifically includes: when the message queue sent by the upstream system is the last message queue in the message queues with the version numbers, the last message queue comprises a last flag bit and a message batch number field;

wherein step S2 specifically includes: the downstream system extracts the message queue with the version number into the database, identifies and counts the version number, establishes a set of one-to-one corresponding version numbers in the database, and stores the message queue with the version number in the set;

reading a last mark and a message batch number field when the last message of the downstream system message queue;

wherein step S3 specifically includes: the downstream system compares the count result of the message queue with the version number to the number of message batches field.

Preferably, the database serves a remote dictionary.

Preferably, step S2 includes the following steps:

step S21: judging whether the database comprises a set corresponding to the version number, and if so, adding the message queue to the set; and if not, the database creates a set corresponding to the version number, and the message queue is stored in the created set.

Preferably, step S3 includes the following steps:

step S31: when the message batch number field is different from the counting result, the downstream system informs the upstream system of sending a failure message to the message queue; step S42: when the message batch number field is the same as the counting result, the downstream system informs the upstream system that the message queue is successfully sent.

Preferably, step S3 further includes the following steps:

step S33: and when the upstream system receives the transmission failure message, the upstream system transmits the message queue again.

Preferably, step S33 further includes counting the number of times of sending the failure message.

Preferably, step S3 further includes the following steps:

step S34: the upstream system is preset with a threshold value, and the threshold value is used for comparing with the sending failure times;

if the number of failed transmissions exceeds the threshold, the upstream system stops transmitting;

if the number of failed transmissions does not exceed the threshold, the upstream system will resend one message queue.

The positive progress effects of the invention are as follows:

1. through the version number, the last flag bit and the message batch number field, the condition that the time point is defined between an upstream system and a downstream system is avoided, and the condition that the CPU pressure of the downstream system is overlarge is also avoided.

2. In the aspect of verification, the development amount of codes is reduced, the complexity of the system is simplified, and the popularization is greatly improved.

Drawings

Fig. 1 is a simplified flowchart of a batch message queue verification method according to embodiment 1 of the present invention.

Fig. 2 is a detailed flowchart of a batch message queue checking method according to embodiment 1 of the present invention.

Fig. 3 is a schematic structural diagram of an electronic device according to embodiment 2 of the present invention.

Detailed Description

The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.

Example 1

Referring to fig. 1, the present embodiment provides a method for checking a batch message queue. The checking method comprises the following steps:

step S1: the upstream system sends a message queue to the downstream system, and the message queue is provided with a version number;

step S2: the downstream system accepts the message queue;

step S3: the downstream system checks the message queue;

in step S1, when the message queue sent by the upstream system is the last message queue in the message queues with the version numbers, the last message queue includes a last flag bit and a message batch number field;

in step S2, the downstream system extracts the message queue with the version number into a database, identifies and counts the version number, establishes a set in the database in one-to-one correspondence with the version number, and stores the message queue with the version number in the set;

when a downstream system accepts the last message of the message queue, reading the last mark and the message batch number field;

in step S3, the downstream system compares the count result of the message queue with the version number with the number of message batches field.

In this embodiment, the upstream system sends a message queue with a version number to the downstream system, the downstream system receives the message queue sent by the upstream system and identifies the version number thereof, and stores the message queue in the database, the database is provided with a plurality of sets, each set corresponds to a version number, and stores the message queue in the set corresponding to the version number, when a message queue is stored in the set of the database, the database will count the corresponding version number, when the message queue sent upstream is the last message queue in the message queue of the corresponding version number, the last message queue includes a last flag bit and a message batch number field, when the last message is sent to the downstream system, the downstream system stores the last message of the version number in the set corresponding to the version number and counts, and simultaneously reading the last flag bit and the message batch field by the downstream system, and comparing the message batch field with the count of the version number by the downstream system.

Specifically, the version number includes but is not limited to "version" 3c8892c2-eba5-48e4-a8f3-0b46536814d0 ".

In particular, the last flag bit includes, but is not limited to, "last",

specifically, the message batch number field includes, but is not limited to, "versionNum".

As a preferred embodiment, the database serves a remote dictionary, and the dictionary service has the advantages of short response time and high read-write performance.

The data table structure of the last message is as follows:

{

"version":"3c8892c2-eba5-48e4-a8f3-0b46536814d0",

"last":true,

"versionNum":100

}

the data structure of the remote dictionary storage is as follows

1. The remote dictionary maintains a message quantity data structure:

{

"key":"count-3c8892c2-eba5-48e4-a8f3-0b46536814d0",

"value":"100",

}

redis save message content data Structure

Referring to the specific steps in figure 2,

step S2 includes the following steps:

step S21: judging whether the database comprises a set corresponding to the version number, if so, adding the message queue to the set; and if not, the database creates a set corresponding to the version number, and the message queue is stored in the created set.

Specifically, step S3 includes the steps of:

step S31: when the message batch number field is different from the counting result, the downstream system informs the upstream system of sending a failure message to the message queue; step S32: when the message batch number field is the same as the counting result, the downstream system informs the upstream system that the message queue is successfully sent.

Step S33: and when the upstream system receives a transmission failure message, the upstream system transmits the message queue again.

The step S33 further includes the downstream system counting the number of times of sending the failure message.

Step S34: a threshold value is preset in the upstream system, and the threshold value is used for comparing with the sending failure times;

if the number of transmission failures exceeds the threshold, the upstream system stops transmitting;

and if the sending failure times do not exceed the threshold value, the upstream system resends the message queue once.

Example 2

Fig. 3 is a schematic structural diagram of an electronic device provided in this embodiment. The electronic device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of bulk message queuing of embodiment 1 when executing the program. The electronic device 30 shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.

As shown in fig. 3, the electronic device 30 may be embodied in the form of a general purpose computing device, which may be, for example, a server device. The components of the electronic device 30 may include, but are not limited to: the at least one processor 31, the at least one memory 32, and a bus 33 connecting the various system components (including the memory 32 and the processor 31).

The bus 33 includes a data bus, an address bus, and a control bus.

The memory 32 may include volatile memory, such as Random Access Memory (RAM)321 and/or cache memory 322, and may further include Read Only Memory (ROM) 323.

Memory 32 may also include a program/utility 325 having a set (at least one) of program modules 324, such program modules 324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The processor 31 executes various functional applications and data processing, such as a processing method of data skew according to embodiment 1 of the present invention, by running the computer program stored in the memory 32.

The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). Such communication may be through input/output (I/O) interfaces 35. Also, model-generating device 30 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via network adapter 36. As shown, network adapter 36 communicates with the other modules of model-generating device 30 via bus 33. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the model-generating device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Example 3

The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of checking a bulk message queue of embodiment 1.

More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.

In a possible implementation, the present invention can also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps of the processing method for data tilting of embodiment 1 when the program product is run on the terminal device.

Where program code for carrying out the invention is written in any combination of one or more programming languages, the program code may be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device or entirely on the remote device.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.