阅读说明：本技术 一种数据处理装置及方法 (data processing device and method ) 是由 陈国涛 孙忠岩 于 2018-06-06 设计创作，主要内容包括：本申请实施例公开了一种数据处理装置及方法,用于提高压缩加密和解密解压的效率,降低系统带宽。本申请实施例提供的数据处理装置中,将压缩组件和加密组件进行桥接,使压缩组件和加密组件之间可以进行数据传输,压缩组件可以将压缩后的压缩数据传输至加密组件,通过加密组件实现对压缩数据的加密,因此在压缩加密过程中,处理器只需对存储模块进行一次读取和一次写入即可。相应的,将解密组件和解压组件进行桥接,使解密组件和解压组件之间可以进行数据传输,解密组件可以将经过解密的解密数据传输至解压组件,通过解压组件实现对解密数据的解压,因此在解密解压过程中,处理器只需对存储模块进行一次读取和一次写入即可。(the embodiment of the application discloses a data processing device and a data processing method, which are used for improving the efficiency of compression, encryption, decryption and decompression and reducing the system bandwidth. In the data processing device that this application embodiment provided, bridge compression subassembly and encryption subassembly, make and to carry out data transmission between compression subassembly and the encryption subassembly, the compression subassembly can transmit the compressed data after the compression to encryption subassembly, realizes the encryption to compressed data through encryption subassembly, consequently in compression encryption process, the treater only need to carry out once to the storage module read and write in can once. Correspondingly, the decryption component and the decompression component are bridged, so that data transmission can be carried out between the decryption component and the decompression component, the decryption component can transmit decrypted decryption data to the decompression component, decompression of the decrypted data is realized through the decompression component, and therefore in the decryption and decompression processes, the processor only needs to read and write the storage module once.)

1. a data processing apparatus, characterized in that the apparatus comprises: data input logic, control logic, compression component, encryption component and data output logic; the control logic is respectively connected with the data input logic, the compression component, the encryption component and the data output logic, the data input logic is connected with the compression component, the compression component is connected with the encryption component, and the encryption component is connected with the data output logic;

The control logic is used for responding to a compression encryption instruction, controlling the data input logic to read data to be processed from a storage module and inputting the data to be processed to the compression component; controlling the compression assembly to compress the data to be processed to obtain compressed data; controlling the encryption component to encrypt the compressed data to obtain encrypted data; and controlling the data output logic to write the encrypted data into the storage module.

2. The apparatus according to claim 1, wherein the compressed encrypted instruction carries a first address and a data length of the data to be processed;

The reading of the data to be processed from the storage module comprises:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

3. The apparatus of claim 1, wherein the controlling the encryption component to encrypt the compressed data comprises:

and if the data volume of the compressed data is greater than or equal to a first threshold value, controlling the encryption component to encrypt the compressed data.

4. the apparatus of claim 1, further comprising key input logic coupled to the encryption component and the control logic;

the controlling the encryption component to encrypt the compressed data comprises:

and controlling the key input logic to read an encryption key from the storage module, and controlling the encryption component to encrypt the compressed data by using the encryption key.

5. The apparatus of any of claims 1-4, further comprising first format transformation logic coupled between the data input logic and the compression component;

the control logic is further configured to control the first format conversion logic to convert the format of the data to be processed from an original format to a preset format, and send the data to be processed in the preset format to the compression component, where the preset format is a format matched with the compression component.

6. the apparatus of claim 5, further comprising second format transformation logic connected between the data output logic and the encryption component;

The control logic is further configured to control the second format conversion logic to convert the format of the encrypted data into the original format, and send the encrypted data in the original format to the data output logic.

7. The apparatus of claim 1, wherein the storage module comprises:

Memory and/or cache pools.

8. a data processing apparatus, characterized in that the apparatus comprises: the device comprises data input logic, control logic, a decryption component, a decompression component and data output logic; the control logic is respectively connected with the data input logic, the decryption component, the decompression component and the data output logic, the data input logic is connected with the decryption component, the decryption component is connected with the decompression component, and the decompression component is connected with the data output logic;

The control logic is used for responding to a decryption decompression instruction, controlling the data input logic to read data to be processed from a storage module and inputting the data to be processed to the decryption component; controlling the decryption component to decrypt the data to be processed to obtain decrypted data; controlling the decompression component to decompress the decrypted data to obtain decompressed data; and controlling the data output logic to write the decompressed data into the storage module.

9. The apparatus according to claim 8, wherein the decryption decompression instruction carries a first address and a data length of the data to be processed;

the reading of the data to be processed from the storage module comprises:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

10. The apparatus of claim 8, further comprising key input logic coupled to the decryption component and the control logic;

The decrypting the data to be processed comprises:

And controlling the key input logic to read a decryption key from the storage module, and controlling the decryption component to decrypt the data to be processed by using the decryption key.

11. The apparatus of any of claims 8-10, further comprising first format transformation logic coupled between the data input logic and the decryption component;

The control logic is further configured to control the first format conversion logic to convert the format of the to-be-processed data from an original format to a preset format, and send the to-be-processed data in the preset format to the decryption component, where the preset format is a format matched with the decryption component.

12. the apparatus of claim 11, further comprising second format transformation logic connected between the data output logic and the decompression component;

The control logic is further configured to control the second format conversion logic to convert the format of the decompressed data into the original format, and send the decompressed data in the original format to the data output logic.

13. The apparatus of claim 8, wherein the storage module comprises:

memory and/or cache pools.

14. a method of data processing, the method comprising:

Responding to the compression encryption instruction, and reading data to be processed from the storage module;

Compressing the data to be processed to obtain compressed data;

encrypting the compressed data to obtain encrypted data;

And writing the encrypted data into the storage module.

15. the method according to claim 14, wherein the compressed encrypted instruction carries a first address and a data length of the data to be processed;

The reading of the data to be processed from the storage module comprises:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

16. The method of claim 14, wherein the encrypting the compressed data comprises:

and if the data volume of the compressed data is greater than or equal to a first threshold value, encrypting the compressed data.

17. The method of claim 14, wherein the encrypting the compressed data comprises:

And reading an encryption key from the storage module, and encrypting the compressed data by using the encryption key.

18. The method according to any one of claims 14 to 17, further comprising:

And converting the format of the data to be processed from the original format into a preset format, wherein the preset format is a format matched with the compression component.

19. The method of claim 18, further comprising:

And transforming the format of the encrypted data into the original format.

20. The method of claim 14, wherein the storage module comprises:

Memory and/or cache pools.

21. A method of data processing, the method comprising:

reading data to be processed from the storage module in response to the decryption decompression instruction;

Decrypting the data to be processed to obtain decrypted data;

Decompressing the decrypted data to obtain decompressed data;

And writing the decompressed data into the storage module.

22. The method according to claim 21, wherein the decryption decompression command carries a first address and a data length of the data to be processed;

The reading of the data to be processed from the storage module comprises:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

23. The method of claim 21, wherein decrypting the data to be processed comprises:

and reading a decryption key from the storage module, and decrypting the data to be processed by using the decryption key.

24. the method according to any one of claims 21-23, further comprising:

and converting the format of the data to be processed from the original format into a preset format, wherein the preset format is a format matched with the decryption component.

25. the method of claim 24, further comprising:

and transforming the format of the decompressed data into the original format.

26. The method of claim 21, wherein the storage module comprises:

Memory and/or cache pools.

27. A data processing apparatus, characterized in that the apparatus comprises: a processor and a memory;

The memory to store instructions;

The processor, configured to execute the instructions in the memory, to perform the method of any of claims 14 to 20.

28. A data processing apparatus, characterized in that the apparatus comprises: a processor and a memory;

the memory to store instructions;

The processor, configured to execute the instructions in the memory, to perform the method of any of claims 21 to 26.

29. a computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 14-20.

30. a computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method of any of claims 21-26.

31. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 14 to 20.

32. a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any of claims 21-26.

Technical Field

The present application relates to the field of data processing, and in particular, to a data processing apparatus and method.

background

With the continuous development of cloud computing and big data, the data volume of global information is increased rapidly year by year, and the demand of compressing data to save storage cost is more and more strong. Meanwhile, encrypting data to achieve security protection of the data is also a very important issue.

In a conventional data processing process, data is generally compressed and then encrypted, and when the data needs to be used, the data is decrypted and then decompressed. In the process of compressing and then encrypting, the compression and the encryption are two independent processes; in the process of decryption before decompression, decryption and decompression are also two independent processes. Therefore, both the processes need to be realized by reading and writing from and into the storage module for storing data for many times, so that the efficiency of compressing, encrypting, decrypting and decompressing the data is reduced, and the system bandwidth is increased.

Disclosure of Invention

the embodiment of the application provides a data processing device and method, which improve the efficiency of compressing, decrypting or decrypting and decompressing data and reduce the system bandwidth.

a first aspect of the present application provides a data processing apparatus, the apparatus comprising: data input logic, control logic, compression component, encryption component and data output logic; the control logic is respectively connected with the data input logic, the compression component, the encryption component and the data output logic, the data input logic is connected with the compression component, the compression component is connected with the encryption component, and the encryption component is connected with the data output logic. The control logic is used for responding to a compression encryption instruction, and the compression encryption instruction is used for indicating that the to-be-processed data stored in the storage module is subjected to compression encryption. The control logic may control the data input logic to read the data to be processed from the storage module, and the storage module may include a memory and/or a cache pool, where the memory stores the data to be processed, the program code, and the like, and the cache pool may store the data to be processed, the compressed data, the encrypted data, and the like. The control logic may control the data input module to input the read data to be processed to the compression assembly. The control component can control the compression component to compress the data to be processed to obtain compressed data; the control module can control the encryption component to encrypt the compressed data to obtain encrypted data; the control module may control the data output logic to write the encrypted data to the storage module.

In the process of compression and encryption, after data to be processed is compressed, the compressed data does not need to be stored in a memory, before the compressed data is encrypted, the compressed data does not need to be read from the memory, only once reading and once writing are carried out on the memory in the whole process, namely, the processor only needs to interact with the storage module twice. In addition, because the compression component and the encryption component are independent components, the encryption of the compressed data can be started after the compressed data reaches a certain data volume, so that the compression and the encryption can be processed in parallel, the compression and encryption efficiency is further improved, and the system bandwidth is reduced.

In some possible designs, the compressed encryption instruction carries the first address and the data length of the data to be processed;

The step of reading the data to be processed from the storage module comprises the following steps:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

the embodiment of the application provides a mode for reading to-be-processed data from a storage module, and the to-be-processed data can be more targeted and more accurate from the storage module according to the first address and the data length of the to-be-processed data.

in some possible designs, controlling the encryption component to encrypt the compressed data includes:

And if the data volume of the compressed data is greater than or equal to the first threshold value, controlling the encryption component to encrypt the compressed data.

In the embodiment of the application, after part of the data to be processed is compressed, the formed compressed data is encrypted through the encryption component, so that the compression operation of the compression component and the encryption operation of the encryption component are processed in parallel, the encryption of the compressed data is not required to be carried out after all the data to be processed are compressed, and the efficiency of compression and encryption is further improved.

In some possible designs, the apparatus further includes key input logic coupled to the encryption component and the control logic;

controlling the encryption component to encrypt the compressed data comprises:

the control key input logic reads the encryption key from the storage module and controls the encryption component to encrypt the compressed data using the encryption key.

in the embodiment of the application, the encryption key can be read through the random input logic, so that the encryption component can encrypt the compressed data by using the encryption key conveniently, and the encrypted data is safer.

in some possible designs, the apparatus further includes a first format transformation logic connected between the data input logic and the compression component;

And the control logic is also used for controlling the first format conversion logic to convert the format of the data to be processed from the original format to a preset format and sending the data to be processed in the preset format to the compression assembly, wherein the preset format is a format matched with the compression assembly.

in the embodiment of the application, the first format conversion logic is used for performing format processing on the data to be processed, and converting the format of the data to be processed from an original format to a preset format, for example, protection information can be added to each piece of data to be processed to protect the integrity of the data, so that the data processing process is safer.

in some possible designs, the apparatus further includes second format transformation logic, the second format transformation logic connected between the data output logic and the encryption component;

And the control logic is also used for controlling the second format conversion logic to convert the format of the encrypted data into the original format and sending the encrypted data in the original format to the data output logic.

In this embodiment of the application, the second format conversion logic is configured to perform format processing on the encrypted data, and convert the format of the encrypted data into an original format, so that the encrypted data can be conveniently used when needed.

in some possible designs, the memory module includes:

Memory and/or cache pools.

in the embodiment of the application, the memory stores data to be processed, program codes and the like, and the cache pool can store the data to be processed, compressed data, encrypted data and the like so as to improve the data reading speed and the data writing speed.

A second aspect of the present application provides another data processing apparatus, comprising: the device comprises data input logic, control logic, a decryption component, a decompression component and data output logic; the control logic is respectively connected with the data input logic, the decryption component, the decompression component and the data output logic, the data input logic is connected with the decryption component, the decryption component is connected with the decompression component, and the decompression component is connected with the data output logic; the control logic is used for responding to a decryption decompression instruction, and the decryption decompression instruction is an instruction used for instructing decryption and decompression of the to-be-processed data stored in the storage module. The control logic may control the data input logic to read the data to be processed from the storage module, and the storage module may include a memory and/or a cache pool, where the memory stores the data to be processed, the program code, and the like, and the cache pool may store the data to be processed, the compressed data, the encrypted data, and the like. The control logic can control the data input module to input the data to be processed into the decryption component; the control logic can control the decryption component to decrypt the data to be processed to obtain decrypted data; the control logic can control the decompression component to decompress the decrypted data to obtain decompressed data; the control logic may control the data output logic to write the decompressed data to the storage module.

In the decryption and decompression process, after data to be processed is decrypted, the decrypted data does not need to be written into the storage module, before the decrypted data is decompressed, the decrypted data does not need to be read from the storage module, the whole process only needs to perform once reading and once writing on the storage module, namely, the processor only needs to interact with the storage module twice, compared with the prior art, the interaction frequency of the processor and the storage module is reduced by half, the decryption and decompression efficiency is improved, and the system bandwidth is reduced by half. In addition, as decryption and decompression are independent processes, decompression of the decrypted data can be started after the decrypted data reaches a certain data volume, so that decryption and decompression can be processed in parallel, the efficiency of decryption and decompression is further improved, and the system bandwidth is reduced.

In some possible designs, the decryption decompression instruction carries the first address and the data length of the data to be processed;

the step of reading the data to be processed from the storage module comprises the following steps:

And reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

The embodiment of the application provides a mode for reading to-be-processed data from a storage module, and the to-be-processed data can be more targeted and more accurate from the storage module according to the first address and the data length of the to-be-processed data.

In some possible designs, the apparatus further includes key input logic coupled to the decryption component and the control logic;

Decrypting the data to be processed comprises:

and the control key input logic reads the decryption key from the storage module and controls the decryption component to decrypt the data to be processed by using the decryption key.

in the embodiment of the application, after partial data to be processed is decrypted, the formed decrypted data can be decompressed through the decompression component, so that the decryption operation of the decryption component and the decompression operation of the decompression component can be processed in parallel, decompression of the decrypted data is not required to be performed after all the data to be processed are decrypted, and the efficiency of decryption and decompression is further improved.

In some possible designs, the apparatus further includes a first format transformation logic connected between the data input logic and the decryption component;

And the control logic is also used for controlling the first format conversion logic to convert the format of the data to be processed from the original format to a preset format and sending the data to be processed in the preset format to the decryption component, wherein the preset format is a format matched with the decryption component.

In this embodiment of the application, the first format conversion logic is configured to convert the format of the data to be processed from an original format to a preset format, and specifically, protection information may be added to each piece of data to be processed to protect integrity of the data.

In some possible designs, the apparatus further includes a second format conversion logic connected between the data output logic and the decompression component;

And the control logic is also used for controlling the second format conversion logic to convert the format of the decompressed data into the original format and sending the decompressed data in the original format to the data output logic.

In this embodiment, the second format conversion logic is configured to convert the format of the decompressed data into an original format, so that the decompressed data can be conveniently used, compressed, or encrypted.

In some possible designs, the memory module includes:

Memory and/or cache pools.

in the embodiment of the application, the memory stores data to be processed, program codes and the like, and the cache pool can store the data to be processed, compressed data, encrypted data and the like so as to improve the data reading speed and the data writing speed.

A third aspect of the present application provides a data processing method, including:

Responding to the compression encryption instruction, and reading data to be processed from the storage module;

Compressing data to be processed to obtain compressed data;

Encrypting the compressed data to obtain encrypted data;

And writing the encrypted data into the storage module.

In some possible designs, the compressed encryption instruction carries the first address and the data length of the data to be processed;

the step of reading the data to be processed from the storage module comprises the following steps:

and reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

In some possible designs, encrypting the compressed data includes:

And if the data volume of the compressed data is greater than or equal to the first threshold value, encrypting the compressed data.

In some possible designs, encrypting the compressed data includes:

And reading the encryption key from the storage module, and encrypting the compressed data by using the encryption key.

In some possible designs, the method further comprises:

And converting the format of the data to be processed from the original format into a preset format, wherein the preset format is a format matched with the compression component.

In some possible designs, the method further comprises:

And converting the format of the encrypted data into the original format.

In some possible designs, the memory module includes:

Memory and/or cache pools.

A fourth aspect of the present application provides another data processing method, including:

Reading data to be processed from the storage module in response to the decryption decompression instruction;

Decrypting the data to be processed to obtain decrypted data;

Decompressing the decrypted data to obtain decompressed data;

and writing the decompressed data into the storage module.

in some possible designs, the decryption decompression instruction carries the first address and the data length of the data to be processed;

The step of reading the data to be processed from the storage module comprises the following steps:

and reading the data to be processed from the storage module according to the first address and the data length of the data to be processed.

in some possible designs, decrypting the data to be processed includes:

and reading the decryption key from the storage module, and decrypting the data to be processed by using the decryption key.

In some possible designs, the method further comprises:

and converting the format of the data to be processed from the original format into a preset format, wherein the preset format is a format matched with the decryption component.

In some possible designs, the method further comprises:

And converting the format of the decompressed data into the original format.

in some possible designs, the memory module includes:

Memory and/or cache pools.

A fifth aspect of the present application provides a data processing apparatus, comprising: a processor and a memory;

A memory to store instructions;

And the processor is used for executing the instructions in the memory and executing the data processing method provided by the third aspect of the application.

A sixth aspect of the present application provides another data processing apparatus, comprising: a processor and a memory;

A memory to store instructions;

And the processor is used for executing the instructions in the memory and executing the data processing method provided by the fourth aspect of the application.

A seventh aspect of the present application provides a computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the data processing method provided by the third aspect of the present application.

an eighth aspect of the present application provides another computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the data processing method provided by the fourth aspect of the present application.

A ninth aspect of the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the data processing method provided by the third aspect of the present application.

A tenth aspect of the present application provides another computer program product containing instructions which, when run on a computer, cause the computer to perform the data processing method provided by the fourth aspect of the present application.

according to the technical scheme, the embodiment of the application has the following advantages:

In a data processing apparatus and method provided in an embodiment of the present application, the data processing apparatus includes: the data processing system comprises data input logic, control logic, a compression assembly, an encryption assembly and data output logic, wherein the control logic is connected with the data input logic, the compression assembly, the encryption assembly and the data output logic, the data input logic is connected with the compression assembly, the compression assembly is connected with the encryption assembly, the encryption assembly is connected with the data output logic, a compression encryption instruction is responded through the control logic, the control data input logic reads data to be processed from a storage module and inputs the data to be processed to the compression assembly, the compression assembly is controlled to compress the data to be processed to obtain compressed data, the encryption assembly is controlled to encrypt the compressed data to obtain encrypted data, and the control data output logic writes the encrypted data into the storage module. Based on the data processing device, in the process of compression and encryption, after data to be processed is compressed, the compressed data does not need to be stored in the memory, before the compressed data is encrypted, the compressed data does not need to be read from the memory, only once reading and once writing are carried out on the memory in the whole process, and multiple times of reading and writing are not needed.

In another data processing apparatus and method provided in an embodiment of the present application, the data processing apparatus includes: the device comprises data input logic, control logic, a decryption component, a decompression component and data output logic; the control logic is respectively connected with the data input logic, the decryption component, the decompression component and the data output logic, the data input logic is connected with the decryption component, the decryption component is connected with the decompression component, and the decompression component is connected with the data output logic; the control logic is used for responding to the decryption decompression instruction, and the control data input logic reads the data to be processed from the storage module and inputs the data to be processed to the decryption component; controlling a decryption component to decrypt the data to be processed to obtain decrypted data; controlling the decompression component to decompress the decrypted data to obtain decompressed data; and controlling the data output logic to write the decompressed data into the storage module. Based on the data processing device, in the decryption and decompression process, after the data to be processed is decrypted, the decrypted data does not need to be written into the storage module, before the decrypted data is decompressed, the decrypted data does not need to be read from the storage module, only once reading and once writing are carried out on the storage module in the whole process, and multiple times of reading and writing are not needed.

drawings

FIG. 1 is a schematic diagram of a data processing flow provided in the prior art;

Fig. 2 is a schematic diagram of a data processing flow provided in an embodiment of the present application;

fig. 3 is a block diagram of a data processing apparatus according to an embodiment of the present application;

Fig. 4 is a schematic diagram of data processing provided in an embodiment of the present application;

fig. 5 is a schematic diagram of a data processing scenario provided in an embodiment of the present application;

Fig. 6 is a block diagram of another data processing apparatus according to an embodiment of the present application;

fig. 7 is a schematic diagram of another data processing scenario provided in an embodiment of the present application;

Fig. 8 is a flowchart of a data processing method according to an embodiment of the present application;

FIG. 9 is a flow chart of another data processing method provided by the embodiments of the present application;

Fig. 10 is a block diagram of a data processing apparatus according to an embodiment of the present application.

Detailed Description

The embodiment of the application provides a data processing device and a data processing method, which are used for improving the efficiency of compression, encryption, decryption and decompression and reducing the system bandwidth.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Currently, for data processing, data is generally compressed and encrypted by a processor, and then the compressed and encrypted data is stored in a storage module; when data is needed to be used, for data processing, a processor usually reads compressed and encrypted data from a storage module, and decrypts and decompresses the compressed and encrypted data to obtain original data.

The storage module is a device for storing data, and in this embodiment, may be used to store data to be processed and/or processed data. The memory module may include a volatile memory, such as a nonvolatile dynamic random access memory (NVRAM), a phase change random access memory (PRAM), a Magnetoresistive Random Access Memory (MRAM), and the like, such as a double data rate synchronous dynamic random access memory (DDR SDRAM), and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, such as a Hard Disk Drive (HDD) or a Solid State Drive (SSD), a flash memory (NOR) or a flash memory (NAND flash memory).

the processor is a device having functions of processing instructions, executing operations, processing data, and the like, and in the embodiment of the present application, is configured to read data from the storage module and process the data, for example, at least one of operations such as compressing, decompressing, encrypting, and decrypting the data may be performed on the data. The processor may be a Central Processing Unit (CPU), a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic devices, a transistor logic device, a hardware component or any combination thereof, or may be a combination for implementing a computing function, such as a combination including one or more microprocessors, a combination of DSPs and microprocessors, and the like.

The processor that currently processes data may include a compression component through which compression of the data is achieved and an encryption component through which encryption of the data is achieved. However, the compression component and the encryption component are independent from each other, and data transmission cannot be performed between them, so for two flows of data compression and data encryption, the processor needs to read data from the storage module and write the data processed by the two flows into the storage module respectively for each flow. In fig. 1, referring to fig. 1(a), fig. 1(a) is a schematic diagram illustrating the interaction between a processor and a memory module in the current implementation of compression encryption. The processor needs to perform the following steps in the compression encryption process: 1) reading data to be processed from the storage module, and compressing the data to be processed; 2) writing the compressed data into a storage module; 3) reading the compressed data from the storage module, and encrypting the compressed data; 4) and writing the encrypted data into the storage module. Thus, the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so that the efficiency of data compression and encryption is low, and the system bandwidth is increased.

Correspondingly, the processor which processes the data at present can further comprise a decryption component and a decompression component, wherein the decryption component is used for decrypting the data, and the decompression component is used for decompressing the data. However, the decryption component and the decompression component are independent from each other and cannot transmit data to each other, so for the two processes of data decryption and data decompression, the processor also needs to read data from the storage module and write the data processed by the two processes into the storage module respectively for each process. In fig. 1, referring to fig. 1(b), fig. 1(b) is a schematic diagram illustrating the interaction between a processor and a storage module in the current implementation of decryption and decompression. The processor needs to perform the following steps in the decryption and decompression processes: 1) reading data to be processed from the storage module, and decrypting the data to be processed; 2) writing the decrypted data into a storage module; 3) reading the decrypted data from the storage module, and decompressing the decrypted data; 4) and writing the decompressed data into a storage module. Therefore, the processor needs to read and write the storage module twice, that is, the processor needs to interact with the storage module four times, so that the efficiency of decrypting and decompressing data is low, and the system bandwidth is increased.

In order to solve the above problem, an embodiment of the present application provides a data processing apparatus, in the apparatus, a compression component and an encryption component are bridged, so that data transmission can be performed between the compression component and the encryption component, the compression component can transmit compressed data after compression to the encryption component, and encryption of the compressed data is realized through the encryption component, so that in a compression and encryption process, a processor only needs to read and write a storage module once. In fig. 2, referring to fig. 2(a), fig. 2(a) is a schematic diagram illustrating interaction between a processor and a storage module when compression encryption is implemented in the data processing method according to the embodiment of the present application. The processor responds to the compression encryption instruction and performs the following steps: 1) reading data to be processed from a storage module, compressing the data to be processed to obtain compressed data, and encrypting the compressed data to obtain encrypted data; 2) and writing the encrypted data into the storage module. In the embodiment of the application, when the compression encryption is realized, the processor only needs to read and write the storage module once, namely, the processor only needs to interact with the storage module twice, and compared with the prior art, the interaction times of the processor and the storage module are reduced by half, so that the compression encryption efficiency is improved, and the system bandwidth is reduced by half.

Correspondingly, in another data processing apparatus provided in this application embodiment, the decryption component and the decompression component are bridged, so that data transmission can be performed between the decryption component and the decompression component, the decryption component can transmit decrypted data to the decompression component, and decompression of the decrypted data is realized through the decompression component, so that in the decryption and decompression processes, the processor only needs to read and write the storage module once. In fig. 2, referring to fig. 2(b), fig. 2(b) is a schematic diagram illustrating interaction between a processor and a storage module when decryption and decompression are implemented in the data processing method according to the embodiment of the present application. The processor responds to the decryption decompression instruction and performs the following steps: 1) reading data to be processed from the storage module, decrypting the data to be processed to obtain decrypted data, and decompressing the decrypted data to obtain decompressed data; 2) and writing the decompressed data into the storage module. In the embodiment of the application, when decryption decompression is realized, the processor only needs to read once and write once, namely, the processor only needs to interact with the storage module twice, and compared with the prior art, the interaction frequency of the processor and the storage module is reduced by half, so that the efficiency of decryption decompression is improved, and the system bandwidth is reduced by half.

referring to fig. 3, a block diagram of a data processing apparatus provided in an embodiment of the present application is shown, where the apparatus includes: data input logic 120, control logic 110, compression component 130, encryption component 140, and data output logic 150. The control logic 110 is connected to the data input logic 120, the compression component 130, the encryption component 140 and the data output logic 150, respectively, the data input logic 120 is connected to the compression component 130, the compression component 130 is connected to the encryption component 140, and the encryption component 140 is connected to the data output logic 150.

Control logic 110 is operable to control data input logic 120, compression component 130, encryption component 140, and data output logic 150 in response to a compress encrypt instruction. The compression encryption instruction is an instruction for instructing compression encryption of data to be processed stored in the storage module 100. At the operating system level, the packed encryption instruction may be sent through the APP, calling the relevant component to execute the packed encryption instruction, such as a Kernel (Kernel) of the driver layer.

In response to the compress encryption instruction, the control logic 110 may control the data input logic 120 to read the data to be processed from the memory module 100, and in actual operation, may call the Zlib APP interface to read the data to be processed from the memory module 100.

The storage module 100 may include a memory in which data to be processed, program codes, and the like are stored, and/or a cache pool in which data to be processed, compressed data, encrypted data, and the like may be stored to improve data reading speed and writing speed.

In this process, the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100, and the data to be processed may be a single file, or multiple files, or a folder including one or more files, or the like. The file type of the data to be processed may be at least one of the types of text, table, mail, database data, and application. The file format of the data to be processed corresponds to the file type of the data to be processed, for example, the format of the text data may be a document (doc), a text file (txt), etc., the format of the table data may be xls, etc., the format of the mail data may be eml, msg, etc., the format of the database data may be db (database), etc., and the format of the application data may be an executable program (EXE), etc.

The data to be processed may also be compressed and/or encrypted data stored in the storage module 100 and needing to be further compressed and encrypted, for example, when the data to be processed is an audio file, the data to be processed may be in a format of compressed data such as moving picture experts group audio layer III (mp 3), and the data to be processed may also be other compressed or encrypted data and have other data formats.

In the compressed encryption instruction, the first address and the data length of the data to be processed may be carried, and at this time, the data input logic 120 may read the data to be processed from the storage module 100 according to the first address and the data length of the data to be processed. When the data to be processed is a plurality of files, the compressed encryption instruction may further carry a plurality of first addresses and a plurality of data lengths corresponding to the first addresses, so that the data input logic 120 reads the data to be processed.

in this embodiment of the application, if the length of the read to-be-processed data is large, optionally, the data processing apparatus may further include a fragmentation logic, and perform fragmentation processing on the to-be-processed data, that is, divide the to-be-processed data into a plurality of data segments, where each data segment is a piece of fragmentation data. Referring to fig. 4, after the data to be processed is fragmented, a plurality of fragmented data can be obtained, and the length of each fragmented data is small, so that the plurality of fragmented data can be compressed and encrypted at the same time, and the compression and encryption efficiency is improved.

after the data input logic 120 reads the data to be processed, the control logic 110 may control the data input logic 120 to input the read data to be processed to the compressing component 130, so that the compressing component 130 compresses the data to be processed.

the data processing apparatus provided in the embodiment of the application may further include a first format conversion logic, where the first format conversion logic is connected between the data input logic 120 and the compression component 130, and is configured to perform format processing on the data to be processed output by the data input logic 120, convert a format of the data to be processed from an original format to a preset format, and the preset format is a format matched with the compression component 130. The data integrity may be protected, for example, by adding protection information to each piece of data to be processed.

The control logic 110 may control the first format conversion logic to perform format conversion on the data to be processed, and after the format conversion, send the processed data to be processed in the preset format to the compression component 130.

After the compression component 130 obtains the data to be processed, the control logic 110 controls the compression component 130 to compress the data to be processed according to a compression algorithm, where data compression is a process of processing the data according to the compression algorithm and representing information by fewer bits than uncompressed data.

the compression component 130 selects a corresponding decompression algorithm according to the identifier of the compression algorithm, where the compression encryption instruction may specifically carry the identifier of the compression algorithm. And compressing the data to be processed to form compressed data, wherein the format of the compressed data can be ZIP, ZLIB and the like. For example, compressed data such as text, tables, mails, database data, and applications may be compressed to obtain compressed data in ZIP format.

After compressing the data to be processed, referring to fig. 4, the compression component 130 may further add a compression header to the compressed data to form compressed data, where the compression header may include a check result for checking the compressed data, so as to implement error detection on the compressed data according to the check result of the compression header, and the compression header may further include a description of the data.

After compressing the data to be processed, the control logic 110 may control the compression component 130 to transmit the compressed data to the encryption component 140 so that the encryption component 140 encrypts the compressed data through an encryption algorithm.

after the encryption component 140 obtains the compressed data, the control logic 110 may control the encryption component 140 to encrypt the compressed data through an encryption algorithm. The data encryption is to process the original plaintext data according to a certain encryption algorithm to make the original plaintext data become unreadable ciphertext, and other people or equipment except a legal receiver are difficult to acquire the plaintext data before encryption, so that the security of data transmission and storage is improved.

common encryption algorithms are, for example, Data Encryption Standard (DES), triple DES, 3DES, message digest algorithm 5 (MD 5), RSA (Rivest-Shamir-Adleman) algorithm, and the like. In the compressed encryption command, the identifier of the encryption algorithm may be carried, and the encryption component 140 may select the corresponding encryption algorithm according to the identifier of the encryption algorithm.

in actual operation, the compressed data may be encrypted by an encryption key, where the encryption key may be a symmetric key, that is, the same key may be used for encryption and decryption, and the encryption key may also be an asymmetric key, that is, the compressed data may be encrypted by a public key and decrypted by a private key. The encryption key may be stored in the storage module 100, and the storage location of the encryption key may also be carried in the compressed encryption instruction. In the embodiment of the present application, the data processing apparatus further includes a key input logic, which is connected to the encryption component 140 and the control logic 110, and is configured to read the encryption key from the storage module 100 according to a storage location of the encryption key. In particular, the control logic 110 controls the key input logic to read the encryption key from the memory module 100 so that the encryption component 140 encrypts the compressed data using the encryption key read by the key input logic.

In the embodiment of the present application, the compressed data that has completed the compression may not be written into the storage module 100, and the compression component 130 may send the compressed data to the encryption component 140 so that the encryption component 140 encrypts the compressed data. On the basis, in order to further improve the efficiency of compression and encryption, after compressing part of the to-be-processed data in the read to-be-processed data, the control logic 110 may further determine whether the data amount of the compressed data that has completed compression is greater than or equal to a first threshold, where the first threshold is the lowest data amount that can be encrypted, and may be 16 bytes (Byte), for example. If the data amount of the compressed data is greater than or equal to the first threshold, the control logic 110 may control the encryption component 140 to encrypt the compressed data that has been compressed, so as to obtain encrypted data.

That is to say, in the embodiment of the present application, after some data to be processed is compressed, the encryption component 140 may encrypt the formed compressed data, so as to implement parallel processing of the compression operation of the compression component 130 and the encryption operation of the encryption component 140, and it is not necessary to wait until all data to be processed is compressed before encrypting the compressed data. For example, if the compression of the first portion of the to-be-processed data is completed to form the first portion of the compressed data, if the data amount of the first portion of the compressed data is 16 bytes and reaches the first threshold 16 bytes, the first portion of the compressed data may be encrypted, and at the same time, the second portion of the to-be-processed data is being compressed, the encryption of the first portion of the compressed data and the compression of the second portion of the to-be-processed data may be performed synchronously, thereby further improving the efficiency of the compression and encryption.

After completing encryption of the compressed data, encrypted data is formed and the encryption component 140 transmits the encrypted data to the data output logic 150. The control logic 110 controls the data output logic 150 to write the formed encrypted data into the storage module 100, where the write location of the encrypted data may be the storage location of the original data to be processed, or may be other unused storage locations. In actual operation, the encrypted data may be written into the storage module 100 by calling an Open Secure Sockets Layer (OpenSLL) interface.

In the data processing apparatus provided in this embodiment of the present application, a second format conversion logic may be further included, where the second format conversion logic is connected between the data output logic 150 and the encryption component 140, after the compressed data is encrypted, the control logic 110 may further control the encryption component 140 to transmit the encrypted data to the second format conversion logic, and control the second format conversion logic 110 to convert the format of the encrypted data into the original format, so that the encrypted data can be conveniently decrypted and decompressed when needed, and transmit the converted encrypted data to the data output module, so that the data output module writes the encrypted data in the original format into the storage module 100.

for convenience of understanding, the data processing apparatus provided in the embodiment of the present application is described with reference to a specific scenario shown in fig. 5. Specifically, the modules in the data processing apparatus may be executed according to the following steps:

501) The APP issues a compressed encryption instruction to the control logic 110.

502) After the control logic 110 responds to the compression encryption command, the control data input module 120 reads the data to be processed from the storage module and sends the read data to be processed to the first format conversion logic.

503) The control logic 110 controls the first format conversion logic to perform format conversion on the data to be processed, so as to form a processing format corresponding to the compression component 130, and transmits the data to be processed after the format conversion to the compression component 130.

504) The control logic 110 controls the compression component 130 to compress the data to be processed according to the compression algorithm to form compressed data, and sends the compressed data to the encryption component 140.

505) the control logic 110 controls the key input module to read the encryption key from the storage module 100 and transmit the read encryption key to the encryption component 140.

506) The control logic 110 controls the encryption component 140 to encrypt the compressed data according to an encryption algorithm, or an encryption algorithm and an encryption key, to form encrypted data, and to send the encrypted data to the second format exchange logic.

507) The control logic 110 controls the second format conversion logic to perform format conversion on the encrypted data, and sends the format-converted encrypted data to the data output module 150.

508) The control logic 110 controls the data output module 150 to write the encrypted data to the memory module 100.

In a data processing apparatus provided in an embodiment of the present application, the data processing apparatus includes: the data processing system comprises data input logic, control logic, a compression assembly, an encryption assembly and data output logic, wherein the control logic is connected with the data input logic, the compression assembly, the encryption assembly and the data output logic, the data input logic is connected with the compression assembly, the compression assembly is connected with the encryption assembly, the encryption assembly is connected with the data output logic, a compression encryption instruction is responded through the control logic, the control data input logic reads data to be processed from a storage module and inputs the data to be processed to the compression assembly, the compression assembly is controlled to compress the data to be processed to obtain compressed data, the encryption assembly is controlled to encrypt the compressed data to obtain encrypted data, and the control data output logic writes the encrypted data into the storage module. In the process of compression and encryption, after data to be processed is compressed, the compressed data does not need to be stored in a memory, before the compressed data is encrypted, the compressed data does not need to be read from the memory, only once reading and once writing are carried out on the memory in the whole process, namely, the processor only needs to interact with the storage module twice. In addition, because the compression component and the encryption component are independent components, the encryption of the compressed data can be started after the compressed data reaches a certain data volume, so that the compression and the encryption can be processed in parallel, the compression and encryption efficiency is further improved, and the system bandwidth is reduced.

in addition, in the aspect of an operating system, in a traditional data processing mode, an APP sends a compression instruction, a ZLib APP interface of a drive layer is called to read data to be processed from a storage module, the data to be processed is compressed, and the compressed data is written into the storage module; then the APP sends an encryption instruction, the OpenSLL interface of the driving layer is called to read the compressed data from the storage module, the compressed data are encrypted, and the encrypted data are written into the storage module. That is to say, the APP needs to send two instructions, that is, a compression instruction and an encryption instruction, and call the Zlib APP interface and the OpenSLL interface respectively, and in this process, it needs to read and write the memory module twice. In the data processing method provided by the embodiment of the application, the APP only needs to send an instruction to the driver layer once, that is, a compression encryption instruction, and the driver layer can implement a compression encryption process on data to be processed. Specifically, the drive layer reads data to be processed from the storage module and compresses the data, then directly encrypts the compressed data, writes the compressed and encrypted data into the storage module after encryption, and only needs to read and write the storage module once in the process.

referring to fig. 6, a block diagram of another data processing apparatus provided in an embodiment of the present application is shown, where the apparatus includes: data input logic 220, control logic 210, decryption component 230, decompression component 240, and data output logic 250. The control logic 210 is connected to the data input logic 220, the decryption component 230, the decompression component 240 and the data output logic 250, respectively, the data input logic 220 is connected to the decryption component 230, the decryption component 230 is connected to the decompression component 240, and the decompression component 240 is connected to the data output logic 250.

Control logic 210 is operable to control data input logic 220, decryption component 230, decompression component 240, and data output logic 250 in response to decrypting the decompression instructions. The decryption decompression instruction is an instruction for instructing decryption decompression of the compressed and encrypted data to be processed stored in the storage module 200. Similar to the packed encrypted instructions, the decrypted decompressed instructions may be sent through the APP, calling the relevant components to execute the decrypted decompressed instructions, such as the Kernel (Kernel) of the driver layer.

in response to the decryption decompression instruction, the control logic 210 may control the data input logic 220 to read the data to be processed from the storage module 200, and in actual operation, may call an OpenSSL interface to read the data to be processed from the storage module 200.

The storage module 200 may include a memory in which data to be processed, program codes, and the like are stored, and/or a cache pool in which data to be processed, compressed data, encrypted data, and the like may be stored to improve data reading speed and writing speed.

In this process, the data to be processed is compressed and encrypted data stored in the storage module 200, and the data to be processed may be a single compressed and encrypted file, a plurality of compressed and encrypted files, or a folder including one or more compressed and encrypted files. The file format of the data to be processed may be ZIP, ZLIB, etc., for example.

in the decryption decompression instruction, the first address and the data length of the data to be processed may be carried, so that the data input logic 220 reads the data to be processed from the storage module 200 according to the first address and the data length of the data to be processed. When the data to be processed is a plurality of compressed and encrypted files, the decryption and decompression instruction may also carry a plurality of first addresses and a plurality of data lengths corresponding to the first addresses.

After the data input logic 220 reads the data to be processed, the control logic 210 controls the data input logic 220 to transmit the read data to be processed to the decryption component 230, so that the decryption component 230 decrypts the data to be processed.

In the data processing apparatus provided in the embodiment of the present application, the data processing apparatus may further include a first format conversion logic, where the first format conversion logic is connected between the data input logic 220 and the decryption component 230, and is configured to convert a format of the data to be processed from an original format to a preset format, where the preset format is a format matched with the decryption component 230, and specifically, protection information may be added to each piece of data to be processed to protect data integrity. The control logic 210 controls the data input logic 220 to transmit the read data to be processed to the first format conversion logic, controls the first format conversion logic to perform format conversion on the data to be processed, and controls the first format conversion logic to send the data to be processed in the preset format to the decryption component 230.

Control logic 210 may control decryption component 230 to decrypt the data to be processed according to a decryption algorithm to form decrypted data. The data decryption is a process of processing the encrypted data through a decryption algorithm to display original content, wherein the decryption algorithm is an inverse algorithm corresponding to the encryption algorithm. The decryption algorithm is corresponding to the encryption algorithm, for example, DES, 3DES, MD5, RSA algorithm, etc., specifically, the compressed encryption instruction may carry an identifier of the decryption algorithm, and the decryption component 230 selects the corresponding decryption algorithm according to the identifier of the decryption algorithm.

In actual operation, if the number of the data to be processed is data encrypted by the encryption key, the data to be processed may be decrypted by the decryption key corresponding to the encryption key of the data to be processed, so as to obtain decrypted data. Specifically, when the key is a symmetric key, the decryption key is the same as the encryption key, and when the key is an asymmetric key, the decryption key is a private key corresponding to the public key.

the decryption key may be stored in the storage module 200, and the decryption key may also be carried in the decryption decompression instruction. In this embodiment, the data control apparatus may further include a key input logic, which is connected to the decryption component 230 and the control logic 210, and is configured to read the decryption key from the storage module 200 according to a storage location of the decryption key. In particular, the control logic 210 may control the key input logic to read the decryption key from the storage module 200, so that the decryption component 230 decrypts the data to be processed by using the decryption key read by the key input logic.

After decrypting the data to be processed, the control logic 210 controls the decryption component 230 to transmit the decrypted data to the decompression component 240, so that the decompression component 240 decompresses the decrypted data through a decompression algorithm.

After the decompression component 240 obtains the decrypted data, the control logic 210 may control the decompression component 240 to decompress the decrypted data via a decompression algorithm. Where data decompression is the inverse of data compression, compressed data may be recovered. For example, compressed data in the ZIP format can be obtained by compressing data such as text, table, video, picture, audio, application program, etc., and data decompression is to restore the data in the ZIP format to the original format, for example, the format of the decompressed text data may be doc, txt, etc., the format of the decompressed mail data may be eml, msg, etc., the format of the decompressed database data may be db, etc., and the format of the decompressed application program data may be EXE, etc. If the data to be processed is obtained by further compressing and encrypting the compressed data, the corresponding decompressed data obtained by decrypting and decompressing the data to be processed is the compressed data, and if the data to be processed is obtained by further compressing and encrypting the encrypted data, the corresponding decompressed data obtained by decrypting and decompressing the data to be processed is the encrypted data.

the decompression algorithm may be, for example, an LZ77 algorithm or a Huffman algorithm, etc. Specifically, the decrypted decompressed instruction may carry an identifier of the decompression algorithm, and the decompression component 240 may select a corresponding decompression algorithm according to the identifier of the decompression algorithm.

In the embodiment of the present application, the decrypted data that has completed decryption may not be written into the storage module 200, and the decryption component 230 may send the decrypted data to the decompression component 240, so that the decompression component 240 decompresses the decrypted data. On this basis, in order to further improve the efficiency of decryption and decompression, after decrypting part of the to-be-processed data in the read to-be-processed data, the control logic 210 may further determine whether the data amount of the decrypted data that has completed decryption is greater than or equal to a second threshold, where the second threshold is the lowest data amount that can be decompressed, and may be 16 bytes, for example. If the data amount of the decrypted data is greater than or equal to the second threshold, the control logic 210 may control the decompression component 240 to decompress the decrypted data that has been decrypted, so as to obtain decompressed data.

That is to say, in the embodiment of the present application, after some data to be processed are decrypted, the decompression component 240 may decompress the formed decrypted data, so that parallel processing of the decryption operation of the decryption component 230 and the decompression operation of the decompression component 240 may be achieved, and it is not necessary to wait until all data to be processed have completed the decryption process and then decompress the decrypted data. For example, if the decryption of the currently existing third part of data to be processed is completed to form the third part of decrypted data, if the data size of the third part of decrypted data is 16Byte and reaches the second threshold 16Byte, the third part of decrypted data may be decompressed, and at the same time, the fourth part of data to be processed is being decrypted, and the decompression of the third part of decrypted data and the decryption of the fourth part of data to be processed may be performed synchronously, thereby further improving the decryption decompression efficiency.

After the decryption data is decompressed, decompressed data is formed, the control logic 210 may control other components to process the formed decompressed data for use, or may control the decompression component 240 to transmit the decompressed data to the data output logic 250, the control logic 210 controls the data output logic 250 to write the formed decompressed data into the storage module 200, and the write-in position of the decompressed data may be a storage position of original data to be processed, or may be another unused storage position.

If the decompressed data is a plurality of fragmented data fragments, the decompressed data can also be merged, and the merged data is written into the storage module 200.

In the data processing apparatus provided in this embodiment of the present application, a second format conversion logic may be further included, where the second format conversion logic is connected between the data output logic 250 and the decompression component 240, after the decryption data is decompressed, the control logic 210 may further control the decompression component 240 to transmit the decompressed data to the second format conversion logic, and the control logic 210 controls the second format conversion logic to convert the format of the decompressed data into the original format, so as to facilitate use, compression, or encryption of the decompressed data. The second format transformation logic transmits the decompressed data after the format transformation to the data output module, so that the data output module writes the decompressed data in the original format into the storage module 200.

for convenience of understanding, the data processing apparatus provided in the embodiment of the present application is described with reference to fig. 7 and a specific scenario. Specifically, the modules in the data processing apparatus may be executed according to the following steps:

701) App issues a decrypt decompress instruction to control logic 210.

702) After the control logic 210 responds to the decryption decompression command, the control data input module 220 reads the to-be-processed data from the storage module 200, and sends the read to-be-processed module to the first format conversion logic.

703) The control logic 210 controls the first format conversion logic to perform format conversion on the data to be processed, so as to form a processing format corresponding to the decryption component 230, and transmits the data to be processed after the format conversion to the decryption component 240.

704) The control logic 210 controls the key input module to read the decryption key from the storage module 200 and to send the read decryption key to the decryption component 230.

705) The control logic 210 controls the decryption component 230 to decrypt the data to be processed according to the decryption algorithm, or the decryption algorithm and the decryption key, to form decrypted data, and sends the decrypted data to the decompression component 240.

706) Control logic 210 controls decompression component 240 to decompress the decrypted data according to a decompression algorithm to form decompressed data and send the decompressed data to the second format conversion logic.

707) The control logic 210 controls the second format conversion logic to perform format conversion on the decompressed data, and sends the decompressed data after format conversion to the data output module 250.

708) the control logic 210 controls the data output module 250 to write the decompressed data to the storage module 200.

In another data processing apparatus and method provided in an embodiment of the present application, the data processing apparatus includes: the device comprises data input logic, control logic, a decryption component, a decompression component and data output logic; the control logic is respectively connected with the data input logic, the decryption component, the decompression component and the data output logic, the data input logic is connected with the decryption component, the decryption component is connected with the decompression component, and the decompression component is connected with the data output logic; the control logic is used for responding to the decryption decompression instruction, and the control data input logic reads the data to be processed from the storage module and inputs the data to be processed to the decryption component; controlling a decryption component to decrypt the data to be processed to obtain decrypted data; controlling the decompression component to decompress the decrypted data to obtain decompressed data; and controlling the data output logic to write the decompressed data into the storage module. In the decryption and decompression process, after data to be processed is decrypted, the decrypted data does not need to be written into the storage module, before the decrypted data is decompressed, the decrypted data does not need to be read from the storage module, the whole process only needs to perform once reading and once writing on the storage module, namely, the processor only needs to interact with the storage module twice, compared with the prior art, the interaction frequency of the processor and the storage module is reduced by half, the decryption and decompression efficiency is improved, and the system bandwidth is reduced by half. In addition, as decryption and decompression are independent processes, decompression of the decrypted data can be started after the decrypted data reaches a certain data volume, so that decryption and decompression can be processed in parallel, the efficiency of decryption and decompression is further improved, and the system bandwidth is reduced.

On the aspect of an operating system, in a traditional data processing mode, an APP sends a decryption instruction, an OpenSLL interface of a drive layer is called to read data to be processed from a storage module, the data to be processed is decrypted, and the decrypted data is written into the storage module; then the APP sends a decompression instruction, the ZLib APP interface of the drive layer is called to read the decrypted data from the storage module, the decrypted data are decompressed, and the decompressed data are written into the storage module. That is to say, the APP needs to send two instructions, that is, a decryption instruction and a decompression instruction, and call the OpenSLL interface and the Zlib APP interface respectively, and in this process, two reads and two writes need to be performed on the memory module. In the data processing method provided by the embodiment of the application, the APP only needs to send a command to the driver layer once, that is, to decrypt and decompress the command, and the driver layer can realize the decryption and decompression process of the data to be processed. Specifically, the drive layer reads the data to be processed from the storage module and decrypts the data, then directly decompresses the decrypted data, writes the decrypted and decompressed data into the storage module after decompression, and only needs to read and write the storage module once in the process.

Referring to fig. 8, a flowchart of a data processing method provided in an embodiment of the present application is shown, where the method may be executed by a processor, and specifically includes the following steps:

And S101, reading the data to be processed from the storage module in response to the compression encryption instruction.

the compression encryption instruction is an instruction for instructing compression encryption of the data to be processed stored in the storage module. At the operating system level, the packed encryption instruction may be sent through the APP, calling the relevant component to execute the packed encryption instruction, such as a Kernel (Kernel) of the driver layer.

The memory module 100 may include a memory in which to-be-processed data, program code, and the like are stored, and/or a cache pool in which to-be-processed data, compressed data, encrypted data, and the like may be stored.

In this step, the data to be processed may be uncompressed and encrypted raw data stored in the storage module 100, or compressed and/or encrypted data stored in the storage module 100 and requiring further compression and encryption. The data to be processed may be a single file, may be a plurality of files, may be a folder including one or more files, and the like.

in the compressed encryption instruction, the first address and the data length of the data to be processed may be carried, so that the data to be processed is read from the storage module 100 according to the first address and the data length of the data to be processed. When the data to be processed is a plurality of files, the compressed encryption instruction may also carry a plurality of first addresses and a plurality of data lengths corresponding to the first addresses.

In this embodiment of the application, if the length of the read to-be-processed data is large, optionally, the to-be-processed data may be firstly subjected to fragmentation processing, that is, the to-be-processed data is divided into a plurality of data segments, and each data segment is a piece of fragmentation data.

and S102, compressing the data to be processed to obtain compressed data.

data compression is the process of processing data according to a compression algorithm to represent information with fewer bits than uncompressed data. The compression method includes the steps of performing compression on a compression instruction, and performing decompression on the compression instruction according to the compression algorithm, wherein the compression instruction includes a compression algorithm identifier, a decompression algorithm identifier, and a compression algorithm identifier. And compressing the data to be processed to form compressed data, wherein the format of the compressed data can be ZIP, ZLIB and the like. For example, compressed data such as text, tables, mails, database data, and applications may be compressed to obtain compressed data in ZIP format.

In the embodiment of the application, format processing may also be performed on the data to be processed, and the format of the data to be processed is converted from an original format to a preset format, where the preset format is a format matched with the component to be compressed. After the data to be processed is compressed, a compression header can be added to the compressed data.

s103, encrypting the compressed data to obtain encrypted data.

data encryption is to process data which is originally in plaintext according to a certain encryption algorithm to enable the data to become unreadable ciphertext, so that other people or equipment except a legal receiver can hardly acquire the data which is in plaintext before encryption, and the security of data transmission and storage is improved accordingly.

common encryption algorithms are, for example, Data Encryption Standard (DES), triple DES, 3DES, message digest algorithm 5 (MD 5), RSA (Rivest-Shamir-Adleman) algorithm, and the like. In the compression encryption instruction, the identifier of the encryption algorithm can be carried, so that the corresponding encryption algorithm can be selected according to the identifier of the encryption algorithm.

In actual operation, the compressed data may be encrypted by an encryption key, where the encryption key may be a symmetric key, that is, the same key may be used for encryption and decryption, and the encryption key may also be an asymmetric key, that is, the compressed data may be encrypted by a public key and decrypted by a private key.

the encryption key can be stored in the storage module, and the storage position of the encryption key can also be carried in the compressed encryption instruction. In the embodiment of the present application, the encryption key may be read from the storage module according to the storage location of the encryption key, so as to encrypt the compressed data by using the encryption key read by the key input logic.

In the embodiment of the present application, the compressed data that has completed compression may not be written into the storage module, but the compressed data may be encrypted. On the basis, in order to further improve the efficiency of compression and encryption, after compressing part of the read data to be processed in the data to be processed, whether the data volume of the compressed data which has been compressed is greater than or equal to the first threshold value can also be judged. And if the data volume of the compressed data is greater than or equal to the first threshold value, encrypting the compressed data which is already compressed to obtain encrypted data. That is to say, in the embodiment of the present application, after some to-be-processed data are compressed, the formed compressed data are encrypted, so that parallel processing of the compression operation and the encryption operation is achieved, it is not necessary to wait until all to-be-processed data are compressed, and then encrypt the compressed data, thereby further improving the efficiency of compression and encryption.

and S104, writing the encrypted data into the storage module.

After the compressed data is encrypted, encrypted data is formed, and the formed encrypted data is written into the storage module, wherein the writing position of the encrypted data can be the storage position of original data to be processed or other unused storage positions. In actual operation, the encrypted data may be written into the storage module by calling an Open Secure Sockets Layer (OpenSLL) interface.

In the embodiment of the application, the format of the encrypted data can be converted into the original format, so that the encrypted data can be conveniently decrypted and decompressed when needed, and the converted encrypted data is written into the storage module.

In the data processing method provided by the embodiment of the application, in response to a compression encryption instruction, to-be-processed data is read from a storage module, the read to-be-processed data is compressed to obtain compressed data, the compressed data is encrypted to obtain encrypted data, and the encrypted data is written into the storage module. In the process of compression and encryption, after data to be processed is compressed, the compressed data does not need to be stored in the storage module, before the compressed data is encrypted, the compressed data does not need to be read from the storage module, only once reading and once writing are carried out on the storage module in the whole process, namely, the processor only needs to interact with the storage module twice. In addition, because the compression and the encryption are independent processes, the encryption of the compressed data can be started after the compressed data reaches a certain data volume, so that the compression and the encryption can be processed in parallel, the compression and the encryption efficiency is further improved, and the system bandwidth is reduced.

Referring to fig. 9, a flowchart of another data processing method provided in the embodiment of the present application is shown, where the method may be executed by a processor, and specifically includes the following steps:

S201, responding to the decryption decompression command, and reading the data to be processed from the storage module.

the decryption decompression instruction is an instruction for instructing decryption and decompression of the compressed and encrypted data to be processed stored in the storage module. Similar to the packed encrypted instructions, the decrypted decompressed instructions may be sent through the APP, calling the relevant components to execute the decrypted decompressed instructions, such as the Kernel (Kernel) of the driver layer.

The storage module 200 may include a memory in which to-be-processed data, program code, and the like are stored, and/or a cache pool in which to-be-processed data, compressed data, encrypted data, and the like may be stored.

In this step, the data to be processed is compressed and encrypted data stored in the storage module, and the data to be processed may be a single compressed and encrypted file, or may be multiple compressed and encrypted files, or may be a folder including one or more compressed and encrypted files.

In the decryption decompression instruction, the first address and the data length of the data to be processed can be carried, so that the data to be processed can be read from the storage module according to the first address and the data length of the data to be processed. When the data to be processed is a plurality of compressed and encrypted files, the decryption and decompression instruction may also carry a plurality of first addresses and a plurality of data lengths corresponding to the first addresses.

S202, decrypting the data to be processed to obtain decrypted data.

The data decryption is a process of processing the encrypted data through a decryption algorithm to display the original content, wherein the decryption algorithm is an inverse algorithm corresponding to the encryption algorithm.

after the data to be processed is obtained, the data to be processed can be decrypted through a decryption algorithm. The decryption algorithm is corresponding to the encryption algorithm, such as DES, 3DES, MD5, RSA algorithm, and the like, and specifically, the compressed encryption instruction may carry an identifier of the decryption algorithm, and the corresponding decryption algorithm is selected according to the identifier of the decryption algorithm.

In actual operation, if the number of the data to be processed is data encrypted by the encryption key, the data to be processed may be decrypted by the decryption key corresponding to the encryption key of the data to be processed, so as to obtain decrypted data. Specifically, when the key is a symmetric key, the decryption key is the same as the encryption key, and when the key is an asymmetric key, the decryption key is a private key corresponding to the public key.

The decryption key may be stored in the storage module, and the decryption decompression instruction may also carry a storage location of the decryption key.

Before decrypting the data to be processed, the format of the data to be processed can be processed, the format of the data to be processed is converted from an original format to a preset format, and the preset format is a format matched with the component for decryption. For example, by adding protection information to each piece of data to be processed to protect the integrity of the data.

And S203, decompressing the decrypted data to obtain decompressed data.

Data decompression is the inverse operation of data compression and can recover compressed data.

After the decrypted data is obtained, the decrypted data may be decompressed according to a decompression algorithm, such as an LZ77 algorithm or a Huffman algorithm. Specifically, the decrypted decompressed command may carry an identifier of the decompression algorithm, and the corresponding decompression algorithm is selected according to the identifier of the decompression algorithm.

In the embodiment of the application, the decrypted data which has completed decryption is not written into the storage module, but is sent to the module for performing data decompression, so as to decompress the decrypted data. On the basis, in order to further improve the efficiency of decryption and decompression, after part of the read data to be processed in the data to be processed is decrypted, whether the data volume of the decrypted data which is decrypted is larger than or equal to a second threshold value can be judged. If the data volume of the decrypted data is greater than or equal to the second threshold, the decrypted data can be decompressed to obtain decompressed data. That is to say, in the embodiment of the present application, after some to-be-processed data are decrypted, the formed decrypted data may be decompressed, so that parallel processing of decryption operation and decompression operation may be implemented, decompression of the decrypted data may not be performed until all to-be-processed data are decrypted, and further improvement of efficiency of decryption and decompression is implemented.

and S204, writing the decompressed data into a storage module.

After the decryption data is decompressed, the formed decompressed data can be processed and used, or the formed decompressed data can be written into the storage module, and the writing position of the decompressed data can be the storage position of the original data to be processed, or other unused storage positions.

If the decompressed data is a plurality of fragmented data fragments, the decompressed data can be merged, and the merged data is written into the storage module. Before the decompressed data is written into the storage module, the format of the decompressed data can be adjusted, and if the format of the decompressed data is a preset format, the format of the decompressed data can be converted into an original format, so that the decompressed data can be conveniently used, compressed or encrypted.

in the data processing method provided by the embodiment of the application, in response to a decryption decompression instruction, data to be processed is read from a storage module, the data to be processed is decrypted first to obtain decrypted data, then the decrypted data is decompressed to obtain decompressed data, and the decompressed data is written into the storage module. In the decryption and decompression process, after data to be processed is decrypted, the decrypted data does not need to be written into the storage module, before the decrypted data is decompressed, the decrypted data does not need to be read from the storage module, the whole process only needs to perform once reading and once writing on the storage module, namely, the processor only needs to interact with the storage module twice, compared with the prior art, the interaction frequency of the processor and the storage module is reduced by half, the decryption and decompression efficiency is improved, and the system bandwidth is reduced by half. In addition, as decryption and decompression are independent processes, decompression of the decrypted data can be started after the decrypted data reaches a certain data volume, so that decryption and decompression can be processed in parallel, the efficiency of decryption and decompression is further improved, and the system bandwidth is reduced.

Next, a data processing apparatus in an embodiment of the present application will be described. Referring to fig. 10, the data processing apparatus 1000 includes:

A receiver 1001, a transmitter 1002, a processor 1003 and a memory 1004 (wherein the number of processors 1003 in the data processing device 1000 may be one or more, one processor is taken as an example in fig. 10). In some embodiments of the present application, the receiver 1001, the transmitter 1002, the processor 1003 and the memory 1004 may be connected by a bus or other means, wherein the connection by the bus is exemplified in fig. 10.

The memory 1004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1003. A portion of the memory 1004 may also include NVRAM. The memory 1004 stores an operating system and operating instructions, executable modules or data structures, or a subset or an expanded set thereof, wherein the operating instructions may include various operating instructions for performing various operations. The operating system may include various system programs for implementing various basic services and for handling hardware-based tasks.

the processor 1003 controls the operation of the terminal device, and the processor 1003 may also be referred to as a CPU. In a particular application, the various components of the data processing device are coupled together by a bus system that may include a power bus, a control bus, a status signal bus, etc., in addition to a data bus. For clarity of illustration, the various buses are referred to in the figures as a bus system.

The method disclosed in the embodiment of the present application may be applied to the processor 1003 or implemented by the processor 1003. The processor 1003 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 1003. The processor 1003 may be a general purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 1004, and the processor 1003 reads the information in the memory 1004, and completes the steps of the method in combination with the hardware thereof.

The receiver 1001 may be used to receive input numeric or character information and to generate signal inputs related to the related settings and function control of the user plane device, the transmitter 1002 may include a display device such as a display screen, and the transmitter 1002 may be used to output numeric or character information through an external interface.

In the embodiment of the present application, the receiver 1001 and the transmitter 1002 are used to implement data transceiving. The processor 1003 is configured to implement data transceiving through the receiver 1001 and the transmitter 1002, and complete a process of one data processing method and a process of another data processing method executed by the foregoing data processing apparatus.

The embodiment of the present application further provides a computer-readable storage medium for storing a program code, where the program code is used to execute any one implementation of the data processing method of the foregoing embodiments.

the embodiment of the present application further provides another computer-readable storage medium for storing a program code, where the program code is used to execute any one implementation of another data processing method in the foregoing embodiments.

The present application further provides a computer program product including instructions, which when run on a computer, causes the computer to execute any one of the implementation manners of the data processing method of the foregoing embodiments.

The present application provides another computer program product including instructions, which when run on a computer, causes the computer to perform any one of the embodiments of the other data processing method of the foregoing embodiments.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

in the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

the integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.