阅读说明：本技术 流量镜像的数据处理方法及其系统 (Data processing method and system of flow mirror image ) 是由 沈渊 曹辉 滕建斌 于 2021-08-13 设计创作，主要内容包括：本发明涉及一种流量镜像的数据处理方法,包括如下步骤：利用镜像端口获取流量数据；识别流量数据的边界,以将流量数据分割为若干数据包；将数据包发送给应用层。本发明有效地解决了传统网络流量数据处理困难的问题,通过优化数据的处理方式,以减小后续分析难度,缩短处理时间,加快处理速度,提升处理效率。(The invention relates to a data processing method of flow mirror image, comprising the following steps: acquiring flow data by using a mirror image port; identifying a boundary of the traffic data to divide the traffic data into a plurality of data packets; and sending the data packet to an application layer. The invention effectively solves the problem of difficult data processing of the traditional network flow, reduces the subsequent analysis difficulty, shortens the processing time, accelerates the processing speed and improves the processing efficiency by optimizing the data processing mode.)

1. A data processing method of flow mirror image is characterized by comprising the following steps:

acquiring flow data by using a mirror image port;

identifying a boundary of the traffic data to segment the traffic data into a number of data packets;

and sending the data packet to an application layer.

2. The data processing method of traffic mirroring according to claim 1, wherein a client in the network layer is connected to a TCP at the server, and when acquiring the traffic data, the method further comprises:

and tracking the connection state of the TCP to obtain a corresponding TCP message, obtaining a serial number in the TCP message, and alarming to an application layer when the serial number is abnormal.

3. The traffic mirrored data processing method according to claim 2, further comprising:

establishing a first cache region and a second cache region corresponding to the client and the server;

and acquiring the flow data of the client and the server and respectively storing the flow data to the first cache region and the second cache region.

4. The traffic mirrored data processing method according to claim 3, further comprising:

determining the connection progress of the TCP according to the flag bit in the TCP message;

and if the connection progress is the second handshake or the first hand waving or the third hand waving of the TCP connection, resetting the first buffer area and the second buffer area.

5. The traffic mirrored data processing method according to claim 1, further comprising:

and converting the flow data into a small-end mode, and further segmenting the flow data into a plurality of data packets.

6. A traffic mirrored data processing system, comprising:

the acquisition module is used for acquiring flow data by using the mirror image port;

the processing module is used for identifying the boundary of the flow data and dividing the flow data into a plurality of data packets; and

and the transmission module is used for sending the data packet to an application layer.

7. The traffic mirrored data processing system of claim 6, wherein a client is connected to a server TCP in a network layer;

the data processing system further comprises a connection tracking module and an alarm module in communication connection with the connection tracking module, wherein the connection tracking module is used for acquiring a TCP message and determining the connection progress of the TCP according to a flag bit in the TCP message, the alarm module is used for acquiring the TCP message in the connection tracking module and judging whether a serial number in the TCP message is abnormal or not, and when the serial number is abnormal, an alarm is given to the application layer.

8. The traffic mirrored data processing system according to claim 7, further comprising a first cache area and a second cache area established corresponding to the client and the server, for storing traffic data of the client and the server, respectively.

9. The traffic mirrored data processing system of claim 8, further comprising a reset module in control connection with the first cache and the second cache, the reset module resetting the first cache and the second cache when the connection tracking module tracks TCP connection progress as a second handshake or a first swipe or a third swipe.

10. The traffic mirrored data processing system of claim 6, further comprising a conversion module communicatively coupled to the acquisition module and the processing module, for converting the traffic data into a small-end mode and sending the small-end mode to the processing module for segmentation.

Technical Field

The invention relates to the technical field of computers, in particular to a data processing method and a data processing system of a flow mirror image.

Background

In the traditional data acquisition method, the port mirror function of the network equipment is started, and the network flow is completely stored in the analysis software for subsequent processing and analysis, however, the network flow introduced into the software is not screened, so that the repeated flow is large, the analysis difficulty is increased, the subsequent data processing speed is reduced, and the efficiency is low.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, and provides a data processing method and a data processing system of a flow mirror image, which solve the problem of difficult data processing of the traditional network flow, reduce the subsequent analysis difficulty, shorten the processing time, accelerate the processing speed and improve the processing efficiency by optimizing the data processing mode.

The technical scheme for realizing the purpose is as follows:

the invention provides a data processing method of flow mirror image, comprising the following steps:

acquiring flow data by using a mirror image port;

identifying a boundary of the traffic data to divide the traffic data into a plurality of data packets;

and sending the data packet to an application layer.

The invention provides a data processing method of flow mirror image, which is characterized in that after flow data is obtained through a mirror image port, the boundary of the flow data is identified, the flow data is divided into a plurality of data packets, and the data packets are further sent to an application layer.

The data processing method of the flow mirror image is further improved in that a client in a network layer is connected with a server TCP, and when the flow data is acquired, the method further comprises the following steps:

and tracking the connection state of the TCP to obtain a corresponding TCP message, obtaining a serial number in the TCP message, and alarming to an application layer when the serial number is abnormal.

The data processing method of the flow mirror image is further improved in that the method further comprises the following steps:

establishing a first cache region and a second cache region corresponding to the client and the server;

and acquiring flow data of the client and the server and respectively storing the flow data to the first cache region and the second cache region.

The data processing method of the flow mirror image is further improved in that the method further comprises the following steps:

determining the connection progress of the TCP according to the zone bit in the TCP message;

and if the connection progress is the second handshake or the first hand waving or the third hand waving of the TCP connection, resetting the first buffer area and the second buffer area.

The data processing method of the flow mirror image is further improved in that the method further comprises the following steps:

and converting the flow data into a small-end mode, and further segmenting the small-end mode to form a plurality of data packets.

The invention also provides a data processing system of the flow mirror image, which comprises:

the acquisition module is used for acquiring flow data by using the mirror image port;

the processing module is used for identifying the boundary of the flow data and dividing the flow data into a plurality of data packets; and

and the transmission module is used for sending the data packet to the application layer.

The data processing system of the flow mirror image is further improved in that a client is connected with a server TCP in a network layer;

the data processing system also comprises a connection tracking module and an alarm module in communication connection with the connection tracking module, wherein the connection tracking module is used for acquiring the TCP message and determining the connection progress of the TCP according to the flag bit in the TCP message, and the alarm module is used for acquiring the TCP message in the connection tracking module, judging whether the serial number in the TCP message is abnormal or not, and alarming to an application layer when the serial number is abnormal.

The data processing system of the flow mirror image is further improved in that the data processing system further comprises a first cache region and a second cache region which are established corresponding to the client and the server and are respectively used for storing flow data of the client and the server.

The data processing system of the flow mirror image is further improved in that the data processing system further comprises a reset module in control connection with the first cache region and the second cache region, and when the connection tracking module tracks that the connection progress of the TCP is handshake for the second time or hand waving for the first time or hand waving for the third time, the reset module resets the first cache region and the second cache region.

The data processing system of the flow mirror image is further improved in that the data processing system of the flow mirror image further comprises a conversion module which is in communication connection with the acquisition module and the processing module and is used for converting the flow data into a small-end mode and sending the small-end mode to the processing module for segmentation.

Drawings

Fig. 1 is a flowchart of a data processing method of traffic mirroring according to the present invention.

Detailed Description

The invention is further described with reference to the following figures and specific examples.

Referring to fig. 1, the invention provides a data processing method and system of a traffic mirror, which identify the boundary of traffic data after the traffic data is acquired by using a mirror port, so as to divide the traffic data into a plurality of data packets, and further send the data packets to an application layer. The data processing method and system of the traffic mirror image according to the present invention will be described with reference to the accompanying drawings.

Referring to fig. 1, a flow chart of the data processing method of the traffic mirroring of the present invention is shown. The data processing method and system of the traffic mirror according to the present invention will be described with reference to fig. 1.

As shown in fig. 1, the present invention provides a data processing method of traffic mirroring, which includes the following steps:

acquiring flow data by using a mirror image port;

identifying a boundary of the traffic data to divide the traffic data into a plurality of data packets;

and sending the data packet to an application layer.

As a preferred embodiment of the present invention, when a client in a network layer is connected to a TCP at a server, and acquires traffic data, the method further includes:

and tracking the connection state of the TCP to obtain a corresponding TCP message, obtaining a serial number in the TCP message, and alarming to an application layer when the serial number is abnormal.

Preferably, whether the sequence number is abnormal or not is judged by tracking and recording the sequence number attribute and the payloaddata attribute of the TcpPacket class, if the sequence number is repeated, it is indicated that a retransmission packet may occur, and if the sequence number is skipped, it is indicated that a packet loss situation may occur, and the corresponding data packet can be deleted, extracted or split according to the actual situation.

Further, the method also comprises the following steps:

establishing a first cache region and a second cache region corresponding to the client and the server;

and acquiring flow data of the client and the server and respectively storing the flow data to the first cache region and the second cache region.

Further, the method also comprises the following steps:

determining the connection progress of the TCP according to the zone bits in the TCP message, namely determining that the connection progress of the TCP is the first hand waving or the third hand waving when the SYN and ACK zone bits of the TCP are both set to be 1 and the second hand shaking of the TCP connection is performed when the SYN and ACK zone bits of the TCP are set to be 1 and the TCP zone bits FIN and ACK are set to be 1;

and if the connection progress is the second handshake or the first hand waving or the third hand waving of the TCP connection, resetting the first buffer area and the second buffer area.

Specifically, the resetting the buffer area includes operations of flushing the unprocessed bytes in the buffer area and resetting the corresponding TCP sequence number.

Preferably, the method further comprises the following steps:

and converting the flow data into a small-end mode, and further segmenting the small-end mode to form a plurality of data packets.

The specific embodiment of the invention is as follows:

acquiring flow data, tracking the TCP connection state of a client and a server in a network layer, acquiring a corresponding TCP message, determining the connection progress of the TCP according to a flag bit in TCP enclosure, and resetting a first cache region and a second cache region when the TCP connection is in second handshake or first hand waving or third hand waving;

the flow data can be converted into a small-end mode according to actual requirements so as to adapt to an X86 architecture;

the head of some flow data defines the packet length or some flow data marks the boundary through characters, the flow data is divided according to the packet length or the character marks to form a plurality of data packets, and the data packets are sent to an application layer;

the method comprises the steps of monitoring a sequence number in a TCP message in the process of obtaining flow data, if the sequence number is repeated or jumped, the situation of packet retransmission, packet sticking or packet loss can occur, at the moment, an alarm is given to an application layer, and the corresponding data packet can be deleted, extracted or split according to the actual situation.

The invention also provides a data processing system of the flow mirror image, which comprises:

the acquisition module is used for acquiring flow data by using the mirror image port;

the processing module is used for identifying the boundary of the flow data and dividing the flow data into a plurality of data packets; and

and the transmission module is used for sending the data packet to the application layer.

Further, a client in the network layer is connected with a server TCP;

the data processing system also comprises a connection tracking module and an alarm module in communication connection with the connection tracking module, wherein the connection tracking module is used for acquiring the TCP message and determining the connection progress of the TCP according to the flag bit in the TCP message, the alarm module is used for acquiring the TCP message in the connection tracking module and judging whether the serial number in the TCP message is abnormal or not, and when the serial number is abnormal, the alarm module gives an alarm to an application layer.

Specifically, the system further comprises a first cache region and a second cache region which are established corresponding to the client and the server and are respectively used for storing the flow data of the client and the server.

Furthermore, the system also comprises a reset module in control connection with the first cache region and the second cache region, and when the connection tracking module tracks that the connection progress of the TCP is the second handshake or the first hand waving or the third hand waving of the TCP connection, the first cache region and the second cache region are reset.

Preferably, the system further comprises a conversion module in communication connection with the acquisition module and the processing module, and the conversion module is used for converting the flow data into a small-end mode and sending the small-end mode to the processing module for segmentation.

The operation mode of the practical implementation of the system provided by the invention is as follows:

the method comprises the steps that a collecting module obtains flow data, a connection tracking module tracks TCP connection states of a client and a server in a network layer, corresponding TCP messages are obtained, connection progress of TCP is determined according to a zone bit in TCP surrounding, and when the TCP connection is in second handshake or first hand waving or third hand waving, a resetting module resets a first cache region and a second cache region;

the flow data can be converted into a small-end mode by using a conversion module according to actual requirements so as to adapt to an X86 architecture;

the head of some flow data defines the packet length or some flow data marks the boundary through characters, the flow data is divided by a processing module according to the packet length or the character marks to form a plurality of data packets, and the data packets are sent to an application layer by a transmission module;

in the process of acquiring the flow data by the acquisition module, the alarm module monitors the serial number in the TCP message, if the serial number is repeated or jumped, the situation of packet retransmission, packet sticking or packet loss may occur, and at this moment, the alarm module gives an alarm to the application layer, and the corresponding data packet can be deleted, extracted or split according to the actual situation.

While the present invention has been described in detail and with reference to the embodiments thereof as illustrated in the accompanying drawings, it will be apparent to one skilled in the art that various changes and modifications can be made therein. Therefore, certain details of the embodiments are not to be interpreted as limiting, and the scope of the invention is to be determined by the appended claims.