阅读说明：本技术 业务数据处理方法、装置、电子设备及存储介质 (Service data processing method and device, electronic equipment and storage medium ) 是由 程勇 陶阳宇 刘舒 于 2021-01-21 设计创作，主要内容包括：本发明提供了一种业务数据处理方法,装置、电子设备、存储介质,方法包括：根据第一样本集合,确定与第一业务方设备相匹配的虚拟样本；确定样本集合交集；确定与第一业务方设备相匹配的第一密钥集合以及与第二业务方设备相匹配的第二密钥集合；利用小批量梯度下降算法对样本集合交集进行虚拟样本删除处理,获得与业务数据处理系统相匹配的训练样本；基于与业务数据处理系统相匹配的训练样本,对业务数据处理系统对应的联邦模型进行训练,确定联邦模型参数由此,在保证数据不交换的情况下,降低计算代价,提升业务数据处理的效率,能够在移动方设备中实现对业务数据的处理,节省用户的等待时间,保证隐私数据不泄露。(The invention provides a business data processing method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: determining a virtual sample matched with the first business side device according to the first sample set; determining a sample set intersection; determining a first key set matched with the first service side equipment and a second key set matched with the second service side equipment; carrying out virtual sample deletion processing on the sample set intersection by using a small batch gradient descent algorithm to obtain a training sample matched with the service data processing system; the method comprises the steps of training a federal model corresponding to a business data processing system based on a training sample matched with the business data processing system, and determining federal model parameters, so that under the condition that data are not exchanged, calculation cost is reduced, the business data processing efficiency is improved, business data can be processed in mobile equipment, user waiting time is saved, and privacy data are not leaked.)

1. A method for processing service data, the method comprising:

acquiring a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment;

determining a virtual sample matched with the first business side device according to the first sample set;

determining a sample set intersection based on the virtual sample matched with the first business side device and a second sample set matched with the second business side device;

determining a first key set matched with the first service side device and a second key set matched with the second service side device;

processing the sample set intersection through the first key set and the second key set, and determining a training sample matched with the business data processing system;

and training a federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determining federal model parameters.

2. The method of claim 1, wherein obtaining a first set of samples matching a first business side device in a business data processing system and a second set of samples matching a second business side device in the business data processing system comprises:

determining a sample set matched with a first service side device in the service data processing system based on the service type of the first service side device;

determining a sample set matched with a second business side device in the business data processing system based on the business type of the second business side device;

and carrying out sample alignment treatment on the sample set matched with the first service side equipment and the sample set matched with the second service side equipment so as to obtain a first sample set matched with the first service side equipment and a second sample set matched with the second service side equipment.

3. The method of claim 1, wherein the determining a virtual sample that matches the first business side device from the first set of samples comprises:

the first service side equipment determines the value parameters and the distribution parameters of the sample IDs in the first sample set;

and generating a virtual sample matched with the first service party device based on the value parameters and the distribution parameters of the sample ID in the first sample set.

4. The method of claim 3, wherein determining a sample set intersection based on the virtual sample matching the first business side device and the second sample set matching the second business side device comprises:

merging the virtual sample and the first sample set to form a first sample set with virtual samples;

traversing a first sample set with virtual samples, and determining an ID set of the virtual samples;

and traversing the second sample set, and determining the sample set intersection of the first sample set with the virtual samples and the second sample set.

5. The method of claim 1, wherein the determining a virtual sample that matches the first business side device from the first set of samples comprises:

triggering a target application process in response to the party device types of the first service party device and the second service party device;

determining, based on the target application process, a set of data intersections of the first set of samples and the second set of samples;

acquiring a first virtual sample set corresponding to the first service side device and a second virtual sample set corresponding to the second service side device through the target application process;

and determining a virtual sample matched with the first service side device through the target application process according to the data intersection set of the first sample set and the second sample set, the first virtual sample set and the second virtual sample set.

6. The method of claim 5, wherein determining a sample set intersection based on the virtual sample matching the first business side device and a second sample set matching the second business side device comprises:

merging the virtual sample and the first sample set to form a first sample set with virtual samples;

traversing a first sample set with virtual samples, and determining an ID set of the virtual samples;

and traversing the second sample set, and determining the sample set intersection of the first sample set with the virtual samples and the second sample set.

7. The method of claim 1, wherein the processing the sample set intersection with the first key set and the second key set to determine the training samples matching the business data processing system comprises:

exchanging different public keys to corresponding service terminals based on the first key set and the second key set so as to obtain initial parameters of a federal model;

determining the number of samples matched with the business data processing system;

and processing the sample set intersection according to the number of the samples, and determining the training samples matched with the business data processing system.

8. The method of claim 1, wherein the training a federated model corresponding to a business data processing system based on the training samples matched to the business data processing system to determine federated model parameters comprises:

substituting the training sample matched with the business data processing system into a loss function corresponding to a federal model corresponding to the business data processing system;

determining a model update parameter corresponding to a federated model corresponding to the business data processing system when the loss function meets a corresponding convergence condition;

and determining model parameters of the federal model based on model update parameters corresponding to the federal model.

9. The method of claim 8, further comprising:

when a federal model corresponding to a business data processing system is trained based on a training sample matched with the business data processing system, residual errors corresponding to virtual samples in the model updating parameters are adjusted through the first business side equipment, so that the influence of the virtual samples on model parameters of the federal model is adjusted.

10. The method of claim 8, further comprising:

when a federal model corresponding to the business data processing system is trained on the basis of a training sample matched with the business data processing system, a target application process is triggered;

and adjusting the residual error corresponding to the virtual sample matched with the model updating parameters based on the target application process so as to adjust the influence of the virtual sample on the model parameters of the federated model.

11. The method of claim 8, further comprising:

and when the business side equipment uses the trained federal model to process business data, setting the virtual sample to zero to adapt to the corresponding business data processing environment.

12. The method according to any one of claims 1-10, further comprising:

and sending the virtual sample, the sample set intersection, the first key set, the second key set and the federal model parameters to a cloud network so as to enable corresponding business side equipment to obtain the virtual sample, the sample set intersection, the first key set, the second key set and the federal model parameters from the cloud network.

13. A service data processing apparatus, characterized in that the apparatus comprises:

the information transmission module is used for acquiring a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment;

the information processing module is used for determining a virtual sample matched with the first business side device according to the first sample set;

the information processing module is used for determining a sample set intersection based on the virtual sample matched with the first business side device and the second sample set matched with the second business side device;

the information processing module is used for determining a first key set matched with the first service side device and a second key set matched with the second service side device;

the information processing module is configured to process the sample set intersection through the first key set and the second key set, and determine a training sample matched with the service data processing system;

and the information processing module is used for training a federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determining federal model parameters.

14. An electronic device, characterized in that the electronic device comprises:

a memory for storing executable instructions;

a processor, configured to execute the executable instructions stored in the memory, and implement the business data processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, implement the business data processing method of any one of claims 1 to 12.

Technical Field

The present invention relates to data processing technologies in cloud networks, and in particular, to a method and an apparatus for processing service data, an electronic device, and a storage medium.

Background

When different business parties share part of business data, secure multi-party calculation needs to be ensured, namely, multiple parties calculate a function result together without revealing input data of each party of the function, and the calculated result is disclosed to one or more parties. In the related art, due to the defect of encryption transmission, privacy data of a user can be frequently leaked, and meanwhile, when a large amount of service data to be processed is faced, the computation complexity of power-mode operation in a traditional exchange encryption function structure is high, the hardware overhead of an encryption process is high, so that the waiting time of the user is long, the hardware use cost is increased, and the realization of service data processing in mobile equipment is not facilitated.

Disclosure of Invention

In view of this, embodiments of the present invention provide a service data processing method, an apparatus, an electronic device, and a storage medium, which can implement processing of a sample set intersection by configuring a corresponding virtual sample determination sample set intersection, determining a training sample matched with a service data processing system, and finally determining a federal model parameter, thereby reducing computation cost, completing a task of determining the federal model parameter, improving efficiency of service data processing, implementing service data processing in a mobile device, saving user latency, and ensuring that private data is not leaked.

The technical scheme of the embodiment of the invention is realized as follows:

the embodiment of the invention provides a service data processing method, which comprises the following steps:

acquiring a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment;

determining a virtual sample matched with the first business side device according to the first sample set;

determining a sample set intersection based on the virtual sample matched with the first business side device and a second sample set matched with the second business side device;

determining a first key set matched with the first service side device and a second key set matched with the second service side device;

processing the sample set intersection through the first key set and the second key set, and determining a training sample matched with the business data processing system;

and training a federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determining federal model parameters.

An embodiment of the present invention further provides a service data processing apparatus, including:

the information transmission module is used for acquiring a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment;

the information processing module is used for determining a virtual sample matched with the first business side device according to the first sample set;

the information processing module is used for determining a sample set intersection based on the virtual sample matched with the first business side device and the second sample set matched with the second business side device;

the information processing module is used for determining a first key set matched with the first service side device and a second key set matched with the second service side device;

the information processing module is configured to process the sample set intersection through the first key set and the second key set, and determine a training sample matched with the service data processing system;

and the information processing module is used for training a federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determining federal model parameters.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for determining a sample set matched with first business side equipment based on the business type of the first business side equipment in the business data processing system;

the information processing module is used for determining a sample set matched with a second business side device based on the business type of the second business side device in the business data processing system;

the information processing module is configured to perform sample alignment processing on the sample set matched with the first service-side device and the sample set matched with the second service-side device, so as to obtain a first sample set matched with the first service-side device and a second sample set matched with the second service-side device.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for determining the value parameters and the distribution parameters of the sample IDs in the first sample set by utilizing first business side equipment;

and the information processing module is used for generating a virtual sample matched with the first business side device based on the value parameters and the distribution parameters of the sample ID in the first sample set.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for merging the virtual sample and the first sample set to form a first sample set with virtual samples;

the information processing module is used for traversing the first sample set with the virtual samples and determining the ID set of the virtual samples;

and the information processing module is used for traversing the second sample set and determining the sample set intersection of the first sample set with the virtual samples and the second sample set.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for responding to the types of the first service side equipment and the second service side equipment and triggering a target application process;

the information processing module is configured to determine a set of data intersections of the first set of samples and the second set of samples based on the target application process;

the information processing module is configured to obtain, through the target application process, a first virtual sample set corresponding to the first service-side device and a second virtual sample set corresponding to the second service-side device;

the information processing module is configured to determine, by the target application process, a virtual sample matched with the first service-side device according to a data intersection set of the first sample set and the second sample set, a first virtual sample set, and a second virtual sample set.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for merging the virtual sample and the first sample set to form a first sample set with virtual samples;

the information processing module is used for traversing the first sample set with the virtual samples and determining the ID set of the virtual samples;

and the information processing module is used for traversing the second sample set and determining the sample set intersection of the first sample set with the virtual samples and the second sample set.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for exchanging different public keys to corresponding business side equipment based on the first key set and the second key set so as to obtain initial parameters of a federal model;

the information processing module is used for determining the number of samples corresponding to the small-batch gradient descent algorithm matched with the service data processing system;

and the information processing module is used for processing the sample set intersection according to the sample number corresponding to the small-batch gradient descent algorithm and determining the training sample matched with the service data processing system.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for substituting the training sample matched with the business data processing system into a loss function corresponding to a federal model corresponding to the business data processing system;

the information processing module is used for determining a model updating parameter corresponding to a federal model corresponding to the business data processing system when the loss function meets corresponding convergence conditions;

and the information processing module is used for determining model parameters of the federal model based on the model update parameters corresponding to the federal model.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for adjusting a residual error corresponding to a virtual sample in the matching of the model update parameters through the first business side equipment when a federal model corresponding to a business data processing system is trained based on a training sample matched with the business data processing system, so as to adjust the influence of the virtual sample on the model parameters of the federal model.

In the above-mentioned scheme, the first step of the method,

the information processing module is used for triggering a target application process when a federal model corresponding to the business data processing system is trained on the basis of a training sample matched with the business data processing system;

and the information processing module is used for adjusting the residual error corresponding to the virtual sample matched with the model updating parameters based on the target application process so as to adjust the influence of the virtual sample on the model parameters of the federated model.

An embodiment of the present invention further provides an electronic device, where the electronic device includes:

a memory for storing executable instructions;

and the processor is used for realizing the business data processing method when the executable instructions stored in the memory are operated.

The present application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes different embodiments and combinations of embodiments provided in various alternative implementations of the business data processing method.

The embodiment of the invention also provides a computer-readable storage medium, which stores executable instructions, and the executable instructions are executed by a processor to realize the business data processing method.

The embodiment of the invention has the following beneficial effects:

the embodiment of the invention obtains a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment; determining a virtual sample matched with the first business side device according to the first sample set; determining a sample set intersection based on the virtual sample matched with the first business side device and a second sample set matched with the second business side device; determining a first key set matched with the first service side device and a second key set matched with the second service side device; processing the sample set intersection through the first key set and the second key set, and determining a training sample matched with the business data processing system; training a federal model corresponding to the business data processing system based on a training sample matched with the business data processing system, and determining federal model parameters; therefore, under the condition that data are not exchanged, the calculation cost is reduced, the task of determining the federal model parameters is completed, the efficiency of processing the business data is improved, the business data can be processed in the mobile device, the waiting time of a user is saved, and the privacy data are not leaked.

Drawings

Fig. 1 is a schematic diagram of a usage environment of a service data processing method according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a service data processing apparatus according to an embodiment of the present invention;

fig. 3 is an optional flowchart of a service data processing method according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a service data processing process of a service data processing method according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a service data processing process of a service data processing method according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a service data processing process of a service data processing method according to an embodiment of the present invention;

fig. 7 is an optional flowchart of a service data processing method according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a service data processing procedure of a service data processing method according to an embodiment of the present invention;

fig. 9 is an optional flowchart of a service data processing method in the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.

1) Business side devices, including but not limited to: the system comprises a common service party device and a special service party device, wherein the common party device is in long connection and/or short connection with a sending channel, and the special service party device is in long connection with the sending channel and can be a server.

2) The client, the carrier in the service side device implementing the specific function, for example, the mobile client (APP) is the carrier of the specific function in the service side device.

3) In response to the condition or state on which the performed operation depends, one or more of the performed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.

4) Federal learning, federal learning is a machine learning framework, and can effectively help a plurality of organizations to carry out data use and machine learning modeling under the condition of meeting the requirements of user privacy protection, data safety and government regulations. The federated learning can effectively solve the data island problem, and the participators can jointly model on the basis of not sharing data, so that the data island can be technically broken, and the cooperation is realized.

5) A Block chain (Block chain) is an encrypted, chained transaction storage structure formed of blocks (blocks).

For example, the header of each block may include hash values of all transactions in the block, and also include hash values of all transactions in the previous block, so as to achieve tamper resistance and forgery resistance of the transactions in the block based on the hash values; newly generated transactions, after being filled into the tiles and passing through the consensus of nodes in the blockchain network, are appended to the end of the blockchain to form a chain growth.

6) A Block chain Network (Block chain Network) that incorporates a new Block into a set of a series of nodes of a Block chain in a consensus manner, where each service device may serve as a different Block chain node in the Block chain Network.

7) The model parameter is a quantity that uses a generic variable to establish a relationship between a function and a variable. In artificial neural networks, the model parameters are typically real matrices.

Fig. 1 is a schematic view of a usage scenario of a service data method according to an embodiment of the present invention, referring to fig. 1, a service side device (including a service side device 10-1 and a service side device 10-2) is provided with a client capable of displaying software of corresponding resource transaction data, for example, a client or a plug-in for performing financial activities or paying through virtual resources, a user may obtain and display resource transaction data through the corresponding client, and trigger a corresponding fraud identification process (for example, a financial payment through a WeChat or a financial loan process through a program in WeChat) in a change process of the virtual resources, in which a data processing device deployed in a server is required to determine a risk of the user, and it is desirable to obtain a processing result of service data in other institutions for auxiliary analysis without obtaining privacy data of nodes of other institutions, to determine the risk level (whether to perform lending) of the target user through the corresponding prediction result; different business side devices may directly connect with business side device 200.

Certainly, the User behavior service data processing apparatus provided by the present invention may be applied to a use environment in which a virtual resource or an entity resource performs financial activities or performs information interaction through an entity financial resource payment environment (including but not limited to various types of entity financial resource change environments, an electronic payment shopping environment, and a use environment in which anti-cheating is performed during e-commerce shopping) or social software, and the financial information of different data sources is usually processed during the financial activities performed on various types of entity financial resources or through virtual resource payment, and finally, target service data of the service data processing system determined by a ranking result of a sample to be tested is presented on a User Interface (UI) of a service side device.

In some embodiments of the invention, the business data processing process may be performed by a computing platform. The computing platform may be a platform provided in the trusted third party device, or may be a platform provided in one of the plurality of data parties or distributed among the plurality of data parties. The computing platform can perform data interaction with various data parties. The multiple business parties (which may be data party servers holding different business data) in fig. 1 may be data parties of the same data category, such as all bank category data parties, or all shopping platform data parties, and so on. The multiple data parties may also be different categories of data parties, such as business party device 10-1 being a shopping platform data party, business party device 10-2 being a lending platform data party, or business party device 10-1 being a data owner of contact information, business party device 10-2 being a service provider, etc. in the above example. In a service data processing scenario, the service data provided by these data parties is usually the same type of service data. For example, in the case where the service device 10-1 is a shopping platform data side and the service device 10-2 is a lending platform data side, if the shopping platform is bound with a payment bank card number and the lending platform is bound with a withdrawal and repayment bank card number, the service data provided by the two sides for service data processing may be the bank card number and transfer information or lending information. If the shopping platform data side and the lending platform data side register the telephone number of the user, the service data provided by the shopping platform data side and the lending platform data side for service data processing can also be the telephone number. In other service scenarios, the service data may also include other data, which is not listed here.

As an example, the service device 200 or the service device 10-1 may be configured to deploy a service data processing apparatus to implement the service data processing method provided by the present invention, so as to obtain a first sample set matching a first service device in a service data processing system and a second sample set matching a second service device in the service data processing system, where the service data processing system includes at least the first service device and the second service device; determining a virtual sample matched with the first business side device according to the first sample set; determining a sample set intersection based on the virtual sample matched with the first business side device and a second sample set matched with the second business side device; determining a first key set matched with the first service side device and a second key set matched with the second service side device; processing the sample set intersection through the first key set and the second key set, and determining a training sample matched with the business data processing system; and training a federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determining federal model parameters.

As will be described in detail below with respect to the structure of the service data processing apparatus according to the embodiment of the present invention, the service data processing apparatus may be implemented in various forms, such as a dedicated service device with a service data processing function, or a server group with a service data processing function, for example, a service information processing process deployed in a service device 10-1, such as the service device 200 in fig. 1. Fig. 2 is a schematic diagram of a composition structure of a service data processing apparatus according to an embodiment of the present invention, and it can be understood that fig. 2 only shows an exemplary structure of the service data processing apparatus, and not a whole structure, and a part of the structure or a whole structure shown in fig. 2 may be implemented as needed.

The service data processing device provided by the embodiment of the invention comprises: at least one processor 201, memory 202, user interface 203, and at least one network interface 204. The various components in the business data processing apparatus are coupled together by a bus system 205. It will be appreciated that the bus system 205 is used to enable communications among the components. The bus system 205 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 205 in fig. 2.

The user interface 203 may include, among other things, a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, or a touch screen.

It will be appreciated that the memory 202 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Memory 202 in embodiments of the present invention is capable of storing data to support operation of a business-side device (e.g., 10-1). Examples of such data include: any computer program, such as an operating system and application programs, for operating on a business side device (e.g., 10-1). The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application program may include various application programs.

In some embodiments, the service data processing apparatus provided in the embodiments of the present invention may be implemented by a combination of hardware and software, and as an example, the service data processing apparatus provided in the embodiments of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to execute the service data processing method provided in the embodiments of the present invention. For example, a processor in the form of a hardware decoding processor may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

As an example that the service data processing apparatus provided by the embodiment of the present invention is implemented by combining software and hardware, the service data processing apparatus provided by the embodiment of the present invention may be directly embodied as a combination of software modules executed by the processor 201, where the software modules may be located in a storage medium, the storage medium is located in the memory 202, and the processor 201 reads executable instructions included in the software modules in the memory 202, and completes the service data processing method provided by the embodiment of the present invention in combination with necessary hardware (for example, including the processor 201 and other components connected to the bus 205).

By way of example, the Processor 201 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor or the like.

As an example of the service data processing apparatus provided by the embodiment of the present invention implemented by hardware, the apparatus provided by the embodiment of the present invention may be implemented by directly using the processor 201 in the form of a hardware decoding processor, for example, by being executed by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components, to implement the service data processing method provided by the embodiment of the present invention.

The memory 202 in the embodiment of the present invention is used to store various types of data to support the operation of the business data processing apparatus. Examples of such data include: any executable instructions for operating on the business data processing apparatus, such as executable instructions, may be included in the executable instructions to implement the program for implementing the business data processing method of the embodiments of the present invention.

In other embodiments, the service data processing apparatus provided by the embodiment of the present invention may be implemented in software, and fig. 2 illustrates the service data processing apparatus stored in the memory 202, which may be software in the form of programs, plug-ins, and the like, and includes a series of modules, as an example of the program stored in the memory 202, the service data processing apparatus may include the following software modules:

the information transmission module 2081 is configured to obtain a first sample set matched with a first service-side device in a service data processing system, and a second sample set matched with a second service-side device in the service data processing system, where the service data processing system at least includes the first service-side device and the second service-side device.

The information processing module 2082 is configured to determine, according to the first sample set, a virtual sample matched with the first service-side device.

The information processing module 2082 is configured to determine a sample set intersection based on the virtual sample matched with the first service-side device and the second sample set matched with the second service-side device.

The information processing module 2082 is configured to determine a first key set that matches the first service device and a second key set that matches the second service device.

The information processing module 2082 is configured to process the sample set intersection through the first key set and the second key set, and determine a training sample matched with the service data processing system.

The information processing module 2082 is configured to train a federal model corresponding to the business data processing system based on a training sample matched with the business data processing system, and determine federal model parameters.

According to the electronic device shown in fig. 2, in one aspect of the present application, the present application also provides a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes different embodiments and combinations of embodiments provided in various alternative implementations of the business data processing method.

Before introducing the service data processing method provided by the present application, a service data processing method in a financial wind control scenario in the related art is described in advance with reference to a service data processing apparatus shown in fig. 2, where in a process of processing service data, due to a large number of service types, each user may have different network data, and some users have tags of some nodes in a network, but often data are not shared with each other for protecting privacy data between users, and for different service side devices, data of users are not exchanged to implement processing of service data, for example: under the bank wind control scene, a bank A hopes to obtain the risk ranking of a current personal credit application customer, wherein the bank A has a historically determined inferior customer, and another bank B has a fund transfer relation of the same customer, and at the moment, the bank A can calculate the risk level of a target customer by using the fund transfer relation of the bank B and a self inferior customer label under the condition that the fund transfer data of the bank B cannot be contacted. While the risk level of a target client can be determined by exchanging user data, the data privacy of the user is disclosed, and the user data outflow is caused.

To solve the above-mentioned drawback, referring to fig. 3, fig. 3 is an optional flowchart of a service data processing method provided in an embodiment of the present invention, and it can be understood that the steps shown in fig. 3 may be executed by various electronic devices operating a service data processing apparatus, for example, a server or a server group that may be used for service data, or a service side device that may be used for a service process. The method specifically comprises the following steps:

step 301: the business data processing device obtains a first sample set matched with a first business side device in the business data processing system and a second sample set matched with a second business side device in the business data processing system.

The service data processing system at least comprises a first service side device and a second service side device; specifically, each service device in the service data processing system may be configured to perform data query on multiple data providers in cooperation with each other for a multiparty join query statement, for example, when multiple data providers perform private data query on multiple providers in cooperation with each other for the multiparty join query statement, or may be a scenario of longitudinal federated learning, where the longitudinal federated learning refers to splitting data sets according to a longitudinal direction (i.e., feature dimensions) and extracting data of the same user and not completely the same user feature from the data sets for training, when users of two data sets overlap more and user features overlap less. This method is called longitudinal federal learning. For example: there are two different institutions, one being a bank in one location and the other being an e-commerce in the same location. Their user population is likely to contain a large proportion of the inhabitants of the site, and therefore the intersection of users is large. However, the bank records the user's income and expense behavior and credit rating, and the e-commerce maintains the user's browsing and purchasing history, so the intersection of the user characteristics is small. Longitudinal federal learning is to aggregate these different features in an encrypted state to enhance model capabilities.

Specifically, data of each data provider is stored in a respective data storage system or cloud server, and original data information required to be disclosed by each provider may be different. The service data processing method provided by the application can exchange the processing results of various privacy data processed by different service side equipment, simultaneously, the original data of each service side equipment is not leaked in the process, and the calculation result is disclosed for each provider, so that each service side equipment can timely and accurately obtain corresponding target service data.

In some embodiments of the present invention, obtaining a first set of samples matching a first business side device in a business data processing system and a second set of samples matching a second business side device in the business data processing system may be accomplished by:

determining a sample set matched with a first service side device in the service data processing system based on the service type of the first service side device; determining a sample set matched with a second business side device in the business data processing system based on the business type of the second business side device; and carrying out sample alignment treatment on the sample set matched with the first service side equipment and the sample set matched with the second service side equipment so as to obtain a first sample set matched with the first service side equipment and a second sample set matched with the second service side equipment. Referring to fig. 4, fig. 4 is a schematic diagram of a business data processing process of the business data processing method according to the embodiment of the present invention, where a participant a and a participant B of a business data processing system respectively have training feature numbersData set D₁And D₂Namely, party a and party B have partial data characteristics, respectively. Participants a and B may extend the data feature dimensions or obtain data label information through vertical federal learning in order to train better models. For example, in two-party vertical federal learning, party a (e.g., an advertising company) and party B (e.g., a social networking platform) cooperate to jointly train one or more deep learning based personalized recommendation models. Wherein party a possesses partial data features, e.g., (X1, X2, …, X40), for a total of 40-dimensional data features; and party B owns another partial data feature, e.g., (X41, X42, …, X100), a total of 60-dimensional data features. The participators A and B jointly have more data characteristics, for example, the A and B data characteristics are 100-dimensional data characteristics together, so the characteristic dimension of the training data is remarkably expanded. For supervised deep learning, participant a and/or participant B also possess label information Y for the training data.

In some embodiments of the invention, one of the two parties does not have feature data, e.g., party a does not have feature data, only tag information.

Before the longitudinal federal learning model training, the participator A and the participator B need to align the training data owned by the participator A and the participator B with the label information, screen out the intersection of the IDs of the training data owned by the participator A and the participator B, and obtain the set D₁And D₂The intersection of the same sample IDs. For example, the feature information XA and XB of the same bank client owned by the participants a and B, respectively, need to be aligned, i.e. they are put together during model training to form a training sample (XA, XB). Wherein, it has no meaning to piece together the characteristic information of different bank customers, can not construct a training sample.

Referring to fig. 5, fig. 5 is a schematic diagram of a business data processing process of the business data processing method according to the embodiment of the present invention, where training sample IDs common to a participant a and a participant B need to be found (this process is also referred to as sample alignment, data alignment, or secure set intersection processing), and common clients of the participant a and the participant B need to be found, that is, clients U1, U2, and U7. For example, the ID of a customer shared by a bank and another home appliance may be generally identified by a hash value of a mobile phone number or an identification number.

Step 302: and the service data processing device determines a virtual sample matched with the first service side equipment according to the first sample set.

Step 303: the business data processing device determines a sample set intersection based on the virtual sample matched with the first business side device and the second sample set matched with the second business side device.

In some embodiments of the present invention, determining a virtual sample matching the first business side device according to the first set of samples may be implemented by:

the first service side equipment determines the value parameters and the distribution parameters of the sample IDs in the first sample set; and generating a virtual sample matched with the first service party device based on the value parameters and the distribution parameters of the sample ID in the first sample set. Wherein the virtual sample and the first sample set can be merged to form a first sample set with virtual samples; traversing a first sample set with virtual samples, and determining an ID set of the virtual samples; and traversing the second sample set, and determining the sample set intersection of the first sample set with the virtual samples and the second sample set. Specifically, as shown in fig. 4 and fig. 5, the participant a randomly generates some virtual sample IDs (and corresponding sample features) according to the values and distribution of the own sample IDs. And the participant A uses the union of the real sample ID set owned by the participant A and the generated virtual sample ID set to perform safe set intersection with the sample ID set of the participant B, so as to obtain an intersection I. The result of this is that the intersection I contains the virtual ID and the real ID of party a. Although both party a and party B know the sample ID information in the intersection I, here the real sample ID is obfuscated with the virtual sample ID, protecting party a's real sample ID from being exactly known by party B.

In some embodiments of the present invention, determining a virtual sample matching the first business side device according to the first sample set may further be implemented by:

triggering a target application process in response to the party device types of the first service party device and the second service party device; determining, based on the target application process, a set of data intersections of the first set of samples and the second set of samples; acquiring a first virtual sample set corresponding to the first service side device and a second virtual sample set corresponding to the second service side device through the target application process; and determining a virtual sample matched with the first service side device through the target application process according to the data intersection set of the first sample set and the second sample set, the first virtual sample set and the second virtual sample set. Merging the virtual sample and the first sample set to form a first sample set with virtual samples; traversing a first sample set with virtual samples, and determining an ID set of the virtual samples; and traversing the second sample set, and determining the sample set intersection of the first sample set with the virtual samples and the second sample set. Specifically, referring to fig. 6 in conjunction with fig. 4 and fig. 5 in the preamble, fig. 6 is a schematic diagram of a service data processing process of the service data processing method provided in the embodiment of the present invention, where a party a and a party B perform secure sample ID set intersection (PSI) by using a third party or a trusted execution environment as a target process to generate a sample ID intersection I₁. The sample ID intersection I₁Is the intersection of the real common sample IDs, excluding the virtual sample IDs.

The third party is referred to herein as party C, as illustrated in fig. 3. At this step, party a and party B may choose to encrypt (or hash) their sample IDs before sending them to party C. If encrypted transmission is chosen, party a and party B need to first perform a key agreement, selecting the same key, e.g. the same RSA public key. Note that if encryption is selected, party C gets the encrypted sample ID and party C cannot decrypt the encrypted sample ID.

The participant C finds the intersection of the received sample ID set sent by the participant a and the sample ID set sent by the participant B, and can complete the comparison simply. Participant C is solving for sample ID intersection I₁Thereafter, ID intersection I is not sent to Party A and Party B₁Only tells the ID intersection I of the party A and the party B₁The number of the elements in the Chinese character. So neither party A nor party B know their common sample ID intersection I₁The specific sample ID of (2). When intersecting set I₁If the number of elements in (1) is too small, the vertical federal learning cannot be performed.

Each of party a and party B generates a virtual sample ID (and corresponding virtual sample characteristics). And the participator A and the participator B use the real sample ID set of the participator A and the generated virtual ID set to carry out intersection solving on the two-party safety set together to obtain an intersection I₂. The sample ID intersection I₂Containing a virtual sample ID. Both Party A and Party B know the sample ID intersection I₂The ID of (3). Because the sample IDs intersect I₂The virtual sample ID is included, so neither party a nor party B knows the exact sample ID of the other party.

In some embodiments of the invention, to guarantee the sample ID intersection I₂The virtual sample ID is contained in the system, and the virtual sample ID generated by the participant A and the participant B needs to be intersected with the real sample ID of the other party. To ensure this, party a and party B may be required to randomly generate virtual sample IDs in the same ID value space. For example, party a and party B may randomly generate cell phone numbers in the same cell phone number segment.

Step 304: the service data processing device determines a first key set matching the first service side device and a second key set matching the second service side device.

Step 305: and the business data processing device processes the sample set intersection through the first key set and the second key set, and determines a training sample matched with the business data processing system.

Wherein, processing the sample set intersection through the first key set and the second key set to determine a training sample matched with the service data processing system includes: exchanging different public keys to corresponding service terminals based on the first key set and the second key set so as to obtain initial parameters of a federal model; and determining the number of samples matched with the business data processing system, processing the sample set intersection according to the number of the samples, and determining training samples matched with the business data processing system. Wherein, processing the sample set intersection according to the sample number corresponding to the small batch gradient descent algorithm comprises selecting batches and small batches, specifically, the participating parties A and B respectively generate respective public key and private key pairs (pk)₁，sk₁) And (pk)₂，sk₂) And sending the public key to the other party. Any party does not reveal its own private key to the other parties. The public key is used for performing additive homomorphic encryption on the intermediate calculation result, for example, performing homomorphic encryption by using a Paillier homomorphic encryption algorithm.

Participation A and B generate random masks R, respectively₂And R₁. Any participant will not disclose any random mask in clear to the other participants. Participants A and B respectively initialize their local model parameters W at random₁And W₂. In the SGD algorithm, only a small batch (mini-batch) of training data is processed in each SGD iteration in order to reduce the amount of computation, speed up the training of the model, and obtain better training effect, for example, each small batch includes 64 training samples. In this case, it is necessary for party a and party B to coordinate the selection of batches and small batches of training samples so that the training samples selected by the two parties in each iteration are aligned.

Step 306: and the business data processing device trains the federal model corresponding to the business data processing system based on the training sample matched with the business data processing system, and determines the federal model parameters.

In some embodiments of the present invention, the federate model corresponding to the business data processing system is trained based on the training sample matched with the business data processing system, and the federate model parameter is determined, which may be implemented by the following steps:

substituting the training sample matched with the business data processing system into a loss function corresponding to a federal model corresponding to the business data processing system; determining a model update parameter corresponding to a federated model corresponding to the business data processing system when the loss function meets a corresponding convergence condition; and determining model parameters of the federal model based on model update parameters corresponding to the federal model. In order to adjust the influence of the virtual sample on the model parameters of the federated model, when the federated model corresponding to the business data processing system is trained based on the training sample matched with the business data processing system, the residual error corresponding to the virtual sample matched with the model update parameters can be adjusted by the first business side device, or a target application process is triggered; and adjusting the residual error corresponding to the virtual sample matched with the model updating parameter based on the target application process. The model training method based on the SGD requires multiple gradient descent iterations, and each iteration can be divided into two stages: (i) forward computing the model output and the residual (also called gradient multiplier); (ii) the gradient of the model parameters by the loss function of the model is propagated backwards and calculated, and the model parameters are updated using the calculated gradient. The above iterations are repeated until a stopping condition is met (e.g., the model parameters converge, or the model loss function converges, or a maximum number of training iterations allowed, or a maximum allowed model training time is reached).

And when the first business side device adjusts the residual error corresponding to the virtual sample matched with the model updating parameter, the participant A and the participant B carry out federal model training on the basis of the sample intersection I, and the participant A is responsible for selecting batch and minor batch (mini-batch) of training samples. To protect participant a's sample IDs, participant a may select some real sample IDs and some virtual sample IDs from the sample intersection I to form oneSmall batches, e.g. 32 virtual samples and 32 real samples, constitute a small batch of 64 samplesWherein m represents the mth small lot. Fig. 7 is a schematic flow chart of an optional process of the business data processing method in the embodiment of the present invention, and referring to fig. 7, when a participant a and a participant B perform federal model training on the basis of the sample intersection, the business data processing may include the following steps:

step 701: and generating a key set matched with different business side devices.

Step 702: public key information is transmitted.

Step 703: participation A and B respectively randomly initialize model parameters W₁And W₂And generates a random mask R₂And R₁。

Step 704: participants A and B are respectively paired with a random mask R₂And R₁And carrying out homomorphic encryption and sending the homomorphic encryption to the other party.

Step 705: party A computationWherein the content of the first and second substances,is the mth batch of training samples owned by party a. Party a generates a random number r1 and sends it in step 705To party B.

Step 706: party A obtains through decryptionParty B decrypts to obtain

Step 707: the participating parties a and B perform calculation processing, respectively.

Thereby can be divided intoAnd

step 708: participant A calculates S, the loss function, the gradient multiplier δ (also called residual)

Wherein, S and the gradient multiplier delta are row vectors, and each element corresponds to each sample in a mini-batch respectively. For example, party a calculates z ═ S₁+S₂And calculating an output of a logistic regression (LogR) model

And gradient operators (also known as residual)

And the participant A only selects a gradient multiplier corresponding to the real sample in one mini-batch to calculate the gradient and update the model parameters. Participant a sets all elements of the gradient multiplier δ that correspond to virtual samples to zero, e.g., participant a generates a row vector δ ═ 0, δ 1,0, δ 3, …, assuming that the first and third samples are virtual samples.

In some embodiments of the invention, party A calculatesWherein N is a small batchThe number of real samples. This is to calculate the average gradient of the small batch. Party A encrypts delta with pk1 to obtain

Party A sends in step 707To party B.

Step 708 Party B computationIt is assumed here thatIs a small batch of data matrix (one sample for each row of the matrix). Wherein r is_BIs a random vector generated by party B.

Step 709: party B sendsTo party a.

Step 710: participant transmission S₂。

In some embodiments of the present invention, with continued reference to FIG. 8, after the steps illustrated in FIG. 7 are performed, the target application process is triggered; based on the target application process, when the residual error corresponding to the virtual sample matched with the model updating parameter is adjusted, the participant A and the participant B are in the sample intersection I₂Is performed on a federal model training basis and is responsible for training batches of samples and selection of small batches by party a. To protect the sample ID of Party A, Party A may select some real sample IDs and some virtual sample IDs from the intersection I to form a small lot, e.g., 32 virtual samples and 32 real samples, into a small lot of 64 samples

Steps 701-710 of the federated model training procedure are fully identical to the steps described in fig. 7, and may be performed iteratively. As shown in fig. 7, in step 711, party a calculates S, the loss function, the gradient multiplier δ (also referred to as the residual). Here z and the gradient multiplier δ are both row vectors, each element corresponding to each sample in a mini-batch, respectively. For example, party a calculates z ═ S1+ S2 and calculates the output of a logistic regression (LogR) model

And gradient operators (also known as residual)The following steps need to be done by party C. The method comprises the following steps:

step 712: participant a sends the gradient multiplier δ to participant C.

Where participant C sets all elements of the received gradient multiplier δ corresponding to the virtual sample to zero, e.g.,it is assumed here that the first and third samples are virtual samples and that party C knows the sample small lotSample ID (either encrypted or hashed). Participant C can cross the intersection I₁To identify virtual samples.

In some embodiments of the invention, participant C calculatesWherein N is a small batchNumber of true samples in (c). Selecting N as a minor batchThe number of real samples in (1) is used for calculating the small batch average gradient, and the data processing speed is improved. Participant C uses its own public key pk3 pairEncrypt, obtain

Step 713: party C sendsTo party a and party B.

Step 714: party A computationAnd transmitTo participant C.

Wherein r is_AIs a random vector generated by party a. Accordingly, party B calculatesAnd transmitTo participant C. Wherein r is_BIs a random vector generated by party B.

Step 715: party A decryption

Step 716: party A sendingTo party B.

Wherein, in step 715,participant C pairDecrypt and sendTo party a. Accordingly, the participant C pairDecrypt and sendTo party B.

In some embodiments of the invention, party A calculates the model loss function versus model parameter W₁For a logistic regression (LogR) model, the model loss function is for the model parameter W₁The gradient of (d) is:the party a updates the model parameters locally: w₁＝W₁-ηg_A. Where η is the learning rate, for example, η ═ 0.01.

Participant B calculates the model loss function vs. model parameter W₂For a logistic regression (LogR) model, the model loss function is for the model parameter W₂The gradient of (d) is:party B updates the model parameters locally: w₁＝W₁-ηg_B. Where η is a learning rate, for example, η is 0.01.

In some embodiments of the invention, participant a and participant B may use different learning rates to update their local model parameters, respectively

In some embodiments of the present invention, when a service side device (service data holder) of a service data processing system migrates or reconfigures the system, a service participant a and B in an embodiment may purchase a service of a blockchain network, and become a corresponding node in the blockchain network through a deployed service side device, so that the service participant a and B may purchase a service of the blockchain network, where a virtual sample, a sample set intersection, a first key set, a second key set, and a federal model parameter and target service data may be sent to the blockchain network, so that the node of the blockchain network fills the virtual sample, the sample set intersection, the first key set, the second key set, and the federal model parameter and the target service data into a new block, and when the consensus for the new block is consistent, the new block is appended to the end of the blockchain. In some embodiments of the present invention, when receiving a data synchronization request of other nodes in the blockchain network, the authority of the other nodes may be verified in response to the data synchronization request; and when the authority of the other nodes passes verification, controlling the current node and the other nodes to perform data synchronization so as to realize that the other nodes acquire the virtual sample, the sample set intersection, the first key set, the second key set, the federal model parameters and the target service data.

In some embodiments of the present invention, the query request may be further analyzed to obtain a corresponding object identifier in response to the query request; acquiring authority information in a target block in a block chain network according to the object identifier; checking the matching of the authority information and the object identification; when the authority information is matched with the object identification, acquiring corresponding virtual samples, sample set intersections, a first key set, a second key set, federal model parameters and target service data in the block chain network; and responding to the query instruction, and pushing the obtained corresponding virtual sample, sample set intersection, first key set, second key set, federal model parameter and target service data to a corresponding client.

The embodiment of the present invention may be implemented by combining a Cloud technology, where the Cloud technology (Cloud technology) is a hosting technology for unifying series resources such as hardware, software, and a network in a wide area network or a local area network to implement calculation, storage, processing, and sharing of data, and may also be understood as a generic term of a network technology, an information technology, an integration technology, a management platform technology, an application technology, and the like applied based on a Cloud computing business model. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, photo-like websites and more portal websites, so cloud technology needs to be supported by cloud computing.

It should be noted that cloud computing is a computing mode, and distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can obtain computing power, storage space and information services as required. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand. As a basic capability provider of cloud computing, a cloud computing resource pool platform, which is called an Infrastructure as a Service (IaaS) for short, is established, and multiple types of virtual resources are deployed in a resource pool and are used by external clients selectively. The cloud computing resource pool mainly comprises: a computing device (which may be a virtualized machine, including an operating system), a storage device, and a network device.

As shown in fig. 1, the data processing method provided in the embodiment of the present invention can be implemented by corresponding cloud devices, for example: different business side devices (including the party device 10-1 and the party device 10-2) are connected with the business side device 200 located at the cloud side through direct connection. It should be noted that the business side device 200 may be a physical device or a virtualized device.

The business data processing method provided by the present application is further described below with reference to different real-time scenarios, wherein cross-industry collaboration scenarios of the financial wind control scenario, such as business side devices, correspond to the credit company a and the bank B, respectively. Wherein, the credit company A receives the loan credit investigation verification of the user as shown in Table 1:

TABLE 1

To further control the risk, credit company a may wish to screen out those users whose deposit is low or unknown before the loan is formally issued, and the user's deposit information is what is outside the credit company a's business.

Meanwhile, bank B owns a user id card set with a credit higher than ten thousand dollars, where S1 is { phone numbers of different users }, refer to table 2.

TABLE 2

Bank B can take further risk control by means of the data of credit company A, i.e. calculate S₁∩S₂And obtaining a final proposal. Specifically referring to fig. 9, fig. 9 is an optional flowchart of the service data processing method provided in the embodiment of the present invention, and the optional flowchart may include the following steps:

step 901: the business data processing device obtains a first sample set matched with a first business side device in the business data processing system and a second sample set matched with a second business side device in the business data processing system.

Step 902: a virtual sample matching the first business party device a is determined.

Step 903: determining the intersection of the A and B sample sets.

Step 904: and exchanging public keys in the key set to determine the training sample.

Step 905: and training a federal model corresponding to the business data processing system, and determining federal model parameters.

Step 906: and deploying the trained federal model to realize service data processing.

The embodiment of the invention obtains a first sample set matched with first business side equipment in a business data processing system and a second sample set matched with second business side equipment in the business data processing system, wherein the business data processing system at least comprises the first business side equipment and the second business side equipment; determining a virtual sample matched with the first business side device according to the first sample set; determining a sample set intersection based on the virtual sample matched with the first business side device and a second sample set matched with the second business side device; determining a first key set matched with the first service side device and a second key set matched with the second service side device; processing the sample set intersection through the first key set and the second key set, and determining a training sample matched with the business data processing system; training a federal model corresponding to the business data processing system based on a training sample matched with the business data processing system, and determining federal model parameters; therefore, under the condition that data are not exchanged, the calculation cost is reduced, the task of determining the federal model parameters is completed, the efficiency of processing the business data is improved, the business data can be processed in the mobile device, the waiting time of a user is saved, and the privacy data are not leaked.

The above description is only exemplary of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.