阅读说明：本技术 一种解决交易数据高并发的处理系统及方法 (Processing system and method for solving high concurrency of transaction data ) 是由 赵浩然 于 2021-09-24 设计创作，主要内容包括：本发明公开了一种解决交易数据高并发的处理系统及方法,属于交易处理技术领域,包括交易数据提交模块,所述交易数据提交模块的输出端与交易数据获取模块的输入端电性连接；本发明能够接收并处理大批量的实时并发交易数据,运行稳定可靠,数据处理结果及时准确,没有出现数据丢失的现象,很好地解决了大量交易大批量周期性浪涌观测数据实时接收、实时处理的难题,提高了该系统的处理效率,同时系统在内还设置有数据处理后的安全检测模块,从多个方面与角度对于交易数据处理过程中易发生的一些问题进行监测,可有效保证对于交易数据处理的安全性,有效解决了交易数据处理中而发生的丢失、泄露等问题,保证系统的安全性,保证应用效果。(The invention discloses a processing system and a method for solving high concurrency of transaction data, belonging to the technical field of transaction processing, and comprising a transaction data submission module, wherein the output end of the transaction data submission module is electrically connected with the input end of a transaction data acquisition module; the invention can receive and process large batch of real-time concurrent transaction data, has stable and reliable operation, timely and accurate data processing result, does not have the phenomenon of data loss, well solves the problems of real-time receiving and real-time processing of large batch of periodic surge observation data of transactions, improves the processing efficiency of the system, is also internally provided with a safety detection module after data processing, monitors some problems which are easy to occur in the transaction data processing process from multiple aspects and angles, can effectively ensure the safety of transaction data processing, effectively solves the problems of loss, leakage and the like in the transaction data processing, ensures the safety of the system and ensures the application effect.)

1. A processing system for solving high concurrency of transaction data, comprising a transaction data submission module, characterized in that: the output end of the transaction data submitting module is electrically connected with the input end of the transaction data acquiring module, the output end of the transaction data acquiring module is electrically connected with the input end of the transaction data processing module (1), the output end of the transaction data processing module (1) is electrically connected with the input end of the transaction data output module, the output end of the transaction data output module is electrically connected with the input end of the load balancing server, the output end of the load balancing server is electrically connected with the input end of the transaction data distributing module (2), the output end of the transaction data distributing module (2) is electrically connected with the input end of the transaction data counting module, the output end of the transaction data counting module is electrically connected with the input end of the transaction data safety monitoring module (3), and the output end of the transaction data safety monitoring module (3) is electrically connected with the input end of the data feedback module, the output end of the data feedback module is electrically connected with the input end of the client.

2. The processing system for solving the high concurrency of transaction data according to claim 1, wherein the load balancing server adopts a Nginx server.

3. The processing system for solving the high concurrency of transaction data according to claim 1, wherein the transaction data processing module (1) comprises a transaction data integration module, an output end of the transaction data integration module is electrically connected with input ends of the useless transaction data eliminating module and the cache data processing module (101), respectively, an output end of the useless transaction data eliminating module is electrically connected with an input end of the transaction data classification module, and an output end of the transaction data classification module is electrically connected with an input end of the transaction data storage module.

4. The processing system for solving the high concurrency of transaction data as claimed in claim 1, wherein the cache data processing module (101) comprises a cache data monitoring module, an output of the cache data monitoring module is electrically connected to an input of a data receiving thread establishing module, an output of the data receiving thread establishing module is electrically connected to an input of an interaction module, an output of the interaction module is electrically connected to an input of a cache data extracting module, and an output of the cache data extracting module is electrically connected to an input of a cache data cleaning module.

5. The processing system for solving the high concurrency of transaction data as claimed in claim 1, wherein the transaction data distribution module (2) comprises an IP mapping relationship configuration module, an output end of the IP mapping relationship configuration module is electrically connected to an input end of a data request topic channel establishment module, an output end of the data request topic channel establishment module is electrically connected to an input end of the transaction data distribution module, an output end of the transaction data distribution module is electrically connected to an input end of a corresponding server, an output end of the corresponding server is electrically connected to an input end of a transaction data processing storage module, and an output end of the transaction data processing storage module is electrically connected to an input end of the data request topic channel establishment module.

6. The system for processing high concurrency of transaction data as claimed in claim 5, wherein the transaction data is stored in the data request subject channel by using a message queue server, and the message queue server is Kafka server.

7. The processing system for solving the high concurrency of transaction data according to claim 1, wherein the transaction data security monitoring module (3) comprises an SQL attack detection module, an output end of the SQL attack detection module is electrically connected with an input end of a background program monitoring module, an output end of the background program monitoring module is electrically connected with an input end of a database vulnerability monitoring module, and an output end of the database vulnerability monitoring module is electrically connected with an input end of a third party personnel operation monitoring module.

8. A processing method for solving high concurrency of transaction data according to any one of claims 1 to 7, comprising the steps of:

s1, submitting external transaction data, and acquiring the submitted transaction data by the system;

s2, processing the acquired transaction data, including integrating the data, eliminating useless transaction data, classifying the transaction data, storing the transaction data and processing some cache data;

s3, in the process of data buffering, firstly monitoring the cached data, establishing a data receiving thread, interacting the cached data and the data receiving thread, and extracting some useless buffered data to quickly clear the data;

s4, outputting the processed data to a load balancing server;

s5, the load balancing server distributes the transaction data request to the corresponding message queue server according to the pre-configured IP mapping relation and the scheduling algorithm;

s6, distributing and respectively processing the transaction data through the corresponding servers, and secondarily storing the processed transaction data;

s7, counting the processed transaction data;

s8, monitoring the safety of the transaction data;

and S9, feeding back the transaction data to the client.

9. The method according to claim 8, wherein in S3, when the processed data just reads the buffer (i), the data in the buffer is read after the corresponding storage operation releases the buffer (i) lock, so as to ensure that all surge data can be processed in parallel timely and efficiently without data loss.

10. The processing method of claim 8, wherein in S9, the client is one or more of a mobile phone APP or a computer APP.

Technical Field

The invention belongs to the technical field of transaction processing, and particularly relates to a processing system and a processing method for solving high concurrency of transaction data.

Background

The transaction is a common exchange mode in daily life, different transactions are carried out all the time and all the time in the world, different transaction objects are different, the traditional transaction mode is gradually replaced by the electronic transaction along with the rapid development of the internet in recent years, and a plurality of transaction data are necessarily in parallel in the electronic transaction process.

Chinese patent discloses (CN202010619132.8) a method and a system for processing transaction data, wherein after receiving target transaction data selected by a user, a terminal device can send object identification information of the target transaction data to a server, and the server performs probability calculation in real time according to object attribute information of the target transaction data to obtain a first processing probability. The terminal device displays the first processing probability, so that the user can know the possibility that the target transaction data is successfully processed finally in real time, and the user can conveniently adjust or confirm the processing request of the target transaction data in a targeted manner. In the embodiment, the first processing probability is displayed on the terminal device in real time, and if the first processing probability is smaller, the processing request can be modified conveniently by the user, so that the possibility that the target transaction data is successfully processed is improved. In the embodiments of the present application, the processing efficiency of the transaction data can be improved, but the processing method does not involve the process of processing the parallel transaction data, and cannot improve the processing efficiency of the transaction data, and cannot improve the transaction efficiency, so as to solve the problem, a processing system and a method for solving the high concurrency of the transaction data are urgently needed.

Disclosure of Invention

The invention aims to: in order to solve the above problems, a processing system and method for solving high concurrency of transaction data are provided.

In order to achieve the purpose, the invention adopts the following technical scheme: a processing system for solving high concurrency of transaction data comprises a transaction data submission module, wherein the output end of the transaction data submission module is electrically connected with the input end of a transaction data acquisition module, the output end of the transaction data acquisition module is electrically connected with the input end of a transaction data processing module, the output end of the transaction data processing module is electrically connected with the input end of a transaction data output module, the output end of the transaction data output module is electrically connected with the input end of a load balancing server, the output end of the load balancing server is electrically connected with the input end of a transaction data distribution module, the output end of the transaction data distribution module is electrically connected with the input end of a transaction data statistic module, the output end of the transaction data statistic module is electrically connected with the input end of a transaction data safety monitoring module, and the output end of the transaction data safety monitoring module is electrically connected with the input end of a data feedback module, the output end of the data feedback module is electrically connected with the input end of the client.

As a further description of the above technical solution:

the load balancing server adopts a Nginx server.

As a further description of the above technical solution:

the transaction data processing module comprises a transaction data integration module, the output end of the transaction data integration module is electrically connected with the input ends of the useless transaction data eliminating module and the cache data processing module respectively, the output end of the useless transaction data eliminating module is electrically connected with the input end of the transaction data classifying module, and the output end of the transaction data classifying module is electrically connected with the input end of the transaction data storing module.

As a further description of the above technical solution:

the buffer data processing module comprises a buffer data monitoring module, the output end of the buffer data monitoring module is electrically connected with the input end of the data receiving thread establishing module, the output end of the data receiving thread establishing module is electrically connected with the input end of the interaction module, the output end of the interaction module is electrically connected with the input end of the buffer data extracting module, and the output end of the buffer data extracting module is electrically connected with the input end of the buffer data cleaning module.

As a further description of the above technical solution:

the transaction data distribution module comprises an IP mapping relation configuration module, the output end of the IP mapping relation configuration module is electrically connected with the input end of the data request subject channel establishment module, the output end of the data request subject channel establishment module is electrically connected with the input end of the transaction data distribution module, the output end of the transaction data distribution module is electrically connected with the input end of the corresponding server, the output end of the corresponding server is electrically connected with the input end of the transaction data processing storage module, and the output end of the transaction data processing storage module is electrically connected with the input end of the data request subject channel establishment module.

As a further description of the above technical solution:

and storing the transaction data into a data request subject channel by using a message queue server, wherein the message queue server adopts a Kafka server.

As a further description of the above technical solution:

the transaction data security monitoring module comprises an SQL attack detection module, the output end of the SQL attack detection module is electrically connected with the input end of a background program monitoring module, the output end of the background program monitoring module is electrically connected with the input end of a database vulnerability monitoring module, and the output end of the database vulnerability monitoring module is electrically connected with the input end of a third-party personnel operation monitoring module.

The invention also discloses a processing method for solving the high concurrency of transaction data, which comprises the following steps:

s1, submitting external transaction data, and acquiring the submitted transaction data by the system;

s2, processing the acquired transaction data, including integrating the data, eliminating useless transaction data, classifying the transaction data, storing the transaction data and processing some cache data;

s3, in the process of data buffering, firstly monitoring the cached data, establishing a data receiving thread, interacting the cached data and the data receiving thread, and extracting some useless buffered data to quickly clear the data;

s4, outputting the processed data to a load balancing server;

s5, the load balancing server distributes the transaction data request to the corresponding message queue server according to the pre-configured IP mapping relation and the scheduling algorithm;

s6, distributing and respectively processing the transaction data through the corresponding servers, and secondarily storing the processed transaction data;

s7, counting the processed transaction data;

s8, monitoring the safety of the transaction data;

and S9, feeding back the transaction data to the client.

As a further description of the above technical solution:

in S3, when the processing data just reads the buffer (i) of the data buffer, the data in the buffer is read after the corresponding storage operation is completed to release the buffer (i) lock, so as to ensure that all surge data can be processed in parallel in time and efficiently without data loss.

As a further description of the above technical solution:

in S9, the client is one or more of a mobile phone APP or a computer APP.

In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:

the system can receive and process a large amount of real-time concurrent transaction data, has stable and reliable operation, timely and accurate data processing result, does not have the phenomenon of data loss, well solves the problems of real-time receiving and real-time processing of a large amount of transaction large-scale periodic surge observation data, improves the processing efficiency of the system, is also provided with the functions of useless transaction data elimination and real-time extraction and cleaning of buffer data, can effectively and quickly eliminate some disordered and irrelevant transaction data, avoids bringing more interference items to the transaction data processing, can realize the effect of real-time cleaning of useless cache, ensures the smooth operation of the system after the large amount of data are flushed, is also internally provided with a safety detection module after the data processing, monitors some problems which are easy to occur in the transaction data processing process from multiple aspects and angles, the method can effectively ensure the safety of transaction data processing, effectively solve the problems of loss, leakage and the like in the transaction data processing, ensure the safety of the system and ensure the application effect.

Drawings

FIG. 1 is a block diagram of a processing system for resolving high concurrency of transaction data.

Fig. 2 is a schematic diagram of a sub-module structure of a transaction data processing module in a processing system for solving high concurrency of transaction data.

FIG. 3 is a sub-module structure diagram of a cache data processing module in a processing system for solving high concurrency of transaction data.

Fig. 4 is a schematic view of a sub-module structure of a transaction data distribution module in a processing system for solving high concurrency of transaction data.

FIG. 5 is a schematic diagram of sub-module structures of a transaction data security monitoring module in a processing system for solving high concurrency of transaction data.

Illustration of the drawings:

1. a transaction data processing module; 101. a cache data processing module; 2. a transaction data distribution module; 3. and the transaction data safety monitoring module.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1-5, the present invention provides a technical solution: a processing system for solving high concurrency of transaction data comprises a transaction data submission module, wherein the output end of the transaction data submission module is electrically connected with the input end of a transaction data acquisition module, the output end of the transaction data acquisition module is electrically connected with the input end of a transaction data processing module 1, the output end of the transaction data processing module 1 is electrically connected with the input end of a transaction data output module, the output end of the transaction data output module is electrically connected with the input end of a load balancing server, the output end of the load balancing server is electrically connected with the input end of a transaction data distribution module 2, the output end of the transaction data distribution module 2 is electrically connected with the input end of a transaction data statistic module, and the output end of the transaction data statistic module is electrically connected with the input end of a transaction data safety monitoring module 3, the output end of the transaction data safety monitoring module 3 is electrically connected with the input end of the data feedback module, the output end of the data feedback module is electrically connected with the input end of the client, and the load balancing server adopts an Nginx server;

the transaction data processing module 1 comprises a transaction data integration module, the output end of the transaction data integration module is respectively electrically connected with the input ends of a useless transaction data eliminating module and a cache data processing module 101, the output end of the useless transaction data eliminating module is electrically connected with the input end of a transaction data classifying module, the output end of the transaction data classifying module is electrically connected with the input end of a transaction data storing module, the cache data processing module 101 comprises a cache data monitoring module, the output end of the cache data monitoring module is electrically connected with the input end of a data receiving thread establishing module, the output end of the data receiving thread establishing module is electrically connected with the input end of an interaction module, the output end of the interaction module is electrically connected with the input end of a cache data extracting module, and the output end of the cache data extracting module is electrically connected with the input end of the cache data cleaning module, the transaction data distribution module 2 comprises an IP mapping relation configuration module, the output end of the IP mapping relation configuration module is electrically connected with the input end of a data request subject channel establishment module, the output end of the data request subject channel establishment module is electrically connected with the input end of the transaction data distribution module, the output end of the transaction data distribution module is electrically connected with the input end of a corresponding server, the output end of the corresponding server is electrically connected with the input end of a transaction data processing storage module, the output end of the transaction data processing storage module is electrically connected with the input end of the data request subject channel establishment module, the transaction data is stored in the data request subject channel and needs to be utilized by a message queue server, the message queue server adopts a Kafka server, the transaction data security monitoring module 3 comprises an SQL attack detection module, the output end of the SQL attack detection module is electrically connected with the input end of the background program monitoring module, the output end of the background program monitoring module is electrically connected with the input end of the database vulnerability monitoring module, and the output end of the database vulnerability monitoring module is electrically connected with the input end of the third-party personnel operation monitoring module;

the invention also discloses a processing method for solving the high concurrency of transaction data, which comprises the following steps:

s1, submitting external transaction data, and acquiring the submitted transaction data by the system;

s2, processing the acquired transaction data, including integrating the data, eliminating useless transaction data, classifying the transaction data, storing the transaction data and processing some cache data;

s3, in the process of data buffering, firstly monitoring the buffered data, establishing a data receiving thread, interacting the buffered data and the data receiving thread, extracting some useless buffered data, quickly cleaning the data, just reading the data buffer area buffer (i) when the processed data is read, waiting for finishing the corresponding storage operation to release the buffer (i) lock, and then reading the data in the buffer area, thereby ensuring that all surge data can be timely and efficiently processed in parallel without data loss;

s4, outputting the processed data to a load balancing server;

s5, the load balancing server distributes the transaction data request to the corresponding message queue server according to the pre-configured IP mapping relation and the scheduling algorithm;

s6, distributing and respectively processing the transaction data through the corresponding servers, and secondarily storing the processed transaction data;

s7, counting the processed transaction data;

s8, monitoring the safety of the transaction data;

and S9, feeding back the transaction data to the client, wherein the client is one or more of a mobile phone APP or a computer APP.

In the embodiment, the system can receive and process a large batch of real-time concurrent transaction data, the operation is stable and reliable, the data processing result is timely and accurate, the phenomenon of data loss does not occur, the problems of real-time receiving and real-time processing of a large amount of transaction large-batch periodic surge observation data are well solved, the processing efficiency of the system is improved, meanwhile, the system is also provided with the functions of useless transaction data elimination and real-time extraction and cleaning of buffer data, the disordered and irrelevant transaction data can be effectively and quickly eliminated, more interference items caused by transaction data processing are avoided, meanwhile, the effect of real-time cleaning of useless cache can be realized, the smoothness of the operation of the system after the large-batch data are flushed is ensured, meanwhile, the system is also internally provided with a safety detection module after the data processing, and problems which are easy to occur in the transaction data processing process are monitored from multiple aspects and angles, the method can effectively ensure the safety of transaction data processing, effectively solve the problems of loss, leakage and the like in the transaction data processing, ensure the safety of the system and ensure the application effect.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.