阅读说明：本技术 一种联邦学习系统 (Federated learning system ) 是由 杨恺 王虎 黄志翔 彭南博 于 2021-03-19 设计创作，主要内容包括：本发明实施例公开了一种联邦学习系统,包括：协作端和至少两个参与端,参与端包括一个标签持有端和至少一个特征持有端,协作端和各参与端之间通信连接,联邦学习系统的训练包括：确定一个参与端作为当前模型更新端；当前模型更新端获取模型更新参数,根据模型更新参数确定加密梯度值,并将加密梯度值发送至协作端,其中,模型更新参数是标签持有端根据上一当前模型更新端的本地模型参数确定的；协作端根据加密梯度值确定模型更新方向,并将模型更新方向发送至当前模型更新端；当前模型更新端根据模型更新方向对本地模型参数进行更新。通过对参与端本地模型的逐个更新,避免了各参与端同时向协作端进行大量的数据传输,提高了联邦学习效率。(The embodiment of the invention discloses a federated learning system, which comprises: the system comprises a collaboration end and at least two participation ends, wherein each participation end comprises a label holding end and at least one characteristic holding end, the collaboration end is in communication connection with each participation end, and the training of the federal learning system comprises: determining a participating end as a current model updating end; the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end, wherein the model updating parameters are determined by the label holding end according to local model parameters of the last current model updating end; the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal; and the current model updating end updates the local model parameters according to the model updating direction. Through updating the local models of the participating terminals one by one, the situation that each participating terminal simultaneously transmits a large amount of data to the cooperation terminal is avoided, and the federal learning efficiency is improved.)

1. A bang learning system, comprising: the system comprises a collaboration end and at least two participation ends, wherein the participation ends comprise a label holding end and at least one characteristic holding end, the collaboration end is in communication connection with the participation ends, and the training of the federal learning system comprises the following steps:

determining a participating end as a current model updating end;

the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end, wherein the model updating parameters are determined by the label holding end according to local model parameters of the last current model updating end;

the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal;

and the current model updating end updates the local model parameters according to the model updating direction.

2. The system according to claim 1, wherein the current model update terminal is a tag holding terminal, the current model update terminal obtains model update parameters, and determines the encryption gradient value according to the model update parameters, including:

and the current model updating end determines the encryption gradient value according to local data and the model updating parameter.

3. The system according to claim 1, wherein the current model update side is a feature holding side, and the current model update side obtains model update parameters, determines an encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the cooperative side, and includes:

and the current model updating end acquires the model updating parameters from the label holding end, and determines the encryption gradient value according to local data and the model updating parameters.

4. The system according to claim 1, wherein the cooperative side determines a model update direction according to the encrypted gradient value, comprising:

and the cooperative terminal decrypts the encrypted gradient value to obtain an original gradient value, and determines the model updating direction according to the original gradient value and a preset step length.

5. The system according to claim 1, wherein the current model update side updates local model parameters according to the model update direction, including;

and the current model updating end takes the difference value between the current local model parameter and the model updating direction as the updated local model parameter.

6. The system of claim 1, further comprising:

and the current model updating end determines an intermediate value according to the updated local model parameter, so that the label holding end updates the model updating parameter according to the intermediate value.

7. The system according to claim 6, wherein the current model update terminal is a tag holding terminal, and the current model update terminal determines an intermediate value according to the updated local model parameter, so that the tag holding terminal updates the model update parameter according to the intermediate value, including:

and the current model updating terminal determines an intermediate value according to the updated local model parameters and updates the model updating parameters according to the intermediate value.

8. The system according to claim 6, wherein the current model update terminal is a feature holding terminal, and the current model update terminal determines an intermediate value according to the updated local model parameter, so that the tag holding terminal updates the model update parameter according to the intermediate value, including:

the current model updating terminal determines an intermediate value according to the updated local model parameters and sends the intermediate value to the label holding terminal;

and the label holding end updates the model updating parameters according to the intermediate value.

9. The system of claim 1, further comprising:

and the cooperation end judges whether a convergence condition is reached according to the encryption gradient values sent by the participating ends, and judges that the federated learning system converges when the convergence condition is reached.

10. The system according to claim 1, wherein the cooperative side determines whether a convergence condition is reached according to the encryption gradient value sent by each of the participating sides, including:

aiming at the encryption gradient value sent by each participating end, the cooperation end judges whether the characteristic value of the encryption gradient value is smaller than a set threshold value or not;

and when the characteristic value of the encryption gradient value of each participating end is smaller than a set threshold value, judging that a convergence condition is reached.

Technical Field

The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a federated learning system.

Background

Federal learning refers to a machine learning framework which can effectively help a plurality of participating terminals (which can represent individuals or organizations) jointly train a model under the requirement of meeting data privacy protection. According to different correlations between samples and characteristics of data held by each participating end of the federal learning, the federal learning can be divided into horizontal federal learning, vertical federal learning, federal transfer learning and the like. Longitudinal federal learning finds wider application in some scenarios. Longitudinal federated learning modeling requires that samples be aligned cryptographically first, followed by cryptographic model training. In the modeling process, each participating end sends information to a third-party cooperative end, namely, each participating end cannot acquire data information and gradient information of the other party, and the data security of each participating end is basically guaranteed. In the process of implementing the invention, the inventor finds that at least the following technical problems exist in the prior art: in the prior art, the data quantity required to be transmitted in each iteration process in the modeling process is large, and the federal learning efficiency is low.

Disclosure of Invention

The embodiment of the invention provides a federated learning system, which aims to reduce the transmission quantity of data during iteration in a modeling process and improve the efficiency of federated learning.

The embodiment of the invention provides a federated learning system, which comprises: the system comprises a collaboration end and at least two participation ends, wherein each participation end comprises a label holding end and at least one characteristic holding end, the collaboration end is in communication connection with each participation end, and the training of the federal learning system comprises:

determining a participating end as a current model updating end;

the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end, wherein the model updating parameters are determined by the label holding end according to local model parameters of the last current model updating end;

the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal;

and the current model updating end updates the local model parameters according to the model updating direction.

Optionally, on the basis of the above scheme, the current model update end is a tag holding end, the current model update end obtains a model update parameter, and determines an encryption gradient value according to the model update parameter, including:

and the current model updating end determines an encryption gradient value according to the local data and the model updating parameters.

Optionally, on the basis of the above scheme, the current model update end is a feature holding end, and the current model update end obtains a model update parameter, determines an encryption gradient value according to the model update parameter, and sends the encryption gradient value to the cooperation end, including:

and the current model updating end acquires the model updating parameters from the label holding end and determines the encryption gradient value according to the local data and the model updating parameters.

Optionally, on the basis of the above scheme, the determining, by the cooperative end, the model update direction according to the encrypted gradient value includes:

and the cooperative terminal decrypts the encrypted gradient value to obtain an original gradient value, and determines a model updating direction according to the original gradient value and a preset step length.

Optionally, on the basis of the above scheme, the current model updating end updates the local model parameters according to the model updating direction, including;

and the current model updating end takes the difference value between the current local model parameter and the model updating direction as the updated local model parameter.

Optionally, on the basis of the above scheme, the method further includes:

and the current model updating end determines an intermediate value according to the updated local model parameters, so that the label holding end updates the model updating parameters according to the intermediate value.

Optionally, on the basis of the above scheme, the current model update end is a tag holding end, and the current model update end determines an intermediate value according to the updated local model parameter, so that the tag holding end updates the model update parameter according to the intermediate value, including:

and the current model updating end determines an intermediate value according to the updated local model parameters and updates the model updating parameters according to the intermediate value.

Optionally, on the basis of the above scheme, the current model update end is a feature holding end, and the current model update end determines an intermediate value according to the updated local model parameter, so that the tag holding end updates the model update parameter according to the intermediate value, including:

the current model updating end determines an intermediate value according to the updated local model parameters and sends the intermediate value to the label holding end;

and the label holding end updates the model updating parameters according to the intermediate value.

Optionally, on the basis of the above scheme, the method further includes:

and the cooperative end judges whether a convergence condition is reached according to the encryption gradient values sent by the participating ends, and judges that the federated learning system converges when the convergence condition is reached.

Optionally, on the basis of the above scheme, the determining, by the cooperative end, whether the convergence condition is reached according to the encryption gradient value sent by each participating end includes:

aiming at the encryption gradient value sent by each participating end, the cooperation end judges whether the characteristic value of the encryption gradient value is smaller than a set threshold value or not;

and when the characteristic values of the encryption gradient values of all the participating ends are smaller than the set threshold value, judging that the convergence condition is reached.

The federal learning system provided by the embodiment of the invention comprises: the system comprises a collaboration end and at least two participation ends, wherein each participation end comprises a label holding end and at least one characteristic holding end, the collaboration end is in communication connection with each participation end, and the training of the federal learning system comprises: determining a participating end as a current model updating end; the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end, wherein the model updating parameters are determined by the label holding end according to local model parameters of the last current model updating end; the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal; the current model updating end updates the local model parameters according to the model updating direction, and through updating the local models of the participating ends one by one, the participating ends are prevented from simultaneously transmitting a large amount of data to the cooperation end, and the federal learning efficiency is improved.

Drawings

FIG. 1 is a flow chart of a training process of a federated learning system provided in an embodiment of the present invention;

fig. 2 is a training flowchart of a bang learning system according to a second embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a training flowchart of a bang learning system according to an embodiment of the present invention. The embodiment can be applied to the situation when the federal learning system is trained, and is particularly suitable for the situation when the longitudinal federal learning system is trained.

In this embodiment, the federal learning system includes a collaboration end and at least two participating ends, where the participating ends include a tag holding end and at least one feature holding end, and the collaboration end is in communication with each participating end. Taking a wind-controlled scene as an example, the tag is only held by one of the participating terminals (which may be called Guest), and each of the other participating terminals only has a partial feature of data (which may be called Host). The effect of the model is improved by cooperation of Guest and Host, and the purpose of reducing risks is achieved. In the process, both the Guest party and the Host party need to ensure the data security of the own party through encryption and a collaboration end. Suppose there isOne of the p participating terminals is Guest, the other p-1 participating terminals are Host, and a third party collaboration terminal Coordinator is used for coordinating the whole process. Setting the local data of the p-th participant end after being registered with the privacy data alignment as x^pThe model parameter is w^p. The overall training data is equivalent to x ═ x (x)¹，x²，…，x^p) And the overall logistic regression model is w ═ (w)¹，w²，…，w^p)。

This embodiment illustrates an iterative process of the federal learning system, and as shown in fig. 1, the training of the federal learning system includes:

and S110, determining a participating end as a current model updating end.

In order to avoid the technical problem of excessive data transmission amount caused by sending data to the cooperative end by each participating end at the same time, in the embodiment, the local models of each participating end are updated one by one, so that the data transmission amount is reduced. Therefore, at a certain iteration, it is necessary to select a participant terminal as the current model update terminal first.

Optionally, a participating end may be randomly selected as the current model updating end, or a model updating sequence may be preset, and the current model updating end is determined according to the preset model updating sequence. For example, assuming that there are p participating peers, each participating peer may be numbered and executed in the order from 1 to p, i.e. p ═ ((t-1) mod p) +1, and one participating peer is determined as the current model update peer according to the order; one participant can also be randomly chosen from 1 to P as the current model update side.

S120, the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end.

In this embodiment, the model update parameters are parameters that are calculated and stored in the tag holding end and are used for updating the local model parameters of the current model update end. When the iteration number is 1, the model updating parameter can be determined by the tag holding end according to the intermediate value of each participating end, and when the iteration number is more than or equal to 1, the model updating parameter can be determined by the tag holding end according to the local model parameter of the last current model updating end. Specifically, the model update parameter is determined according to the intermediate value of the last current model update end.

Optionally, when training is started, the local model of each participating end generates initial model parameters, for each participating end, an intermediate value is calculated according to the initial model parameters of the local model of each participating end, and each feature holding end transmits the ciphertext to the tag holding end through an addition homomorphic encryption algorithm. The label holding end can obtain intermediate values of all the feature holding ends, and model updating parameters are calculated according to the principle of addition homomorphic encryption, so that the initial current model updating end updates local model parameters according to the model updating parameters. Illustratively, the intermediate value in the participating end may be by u_p(0)＝w^p(0)^Tx^pIs calculated to obtain, wherein u_p(0) Is the median of the p-th participating end, w^p(0) For the p-th participating end's local model parameter, x^pIs the local training data of the p-th participant. The model update parameters may be passedCalculated, wherein [ [ d (0) ]]]Update parameters for the model, y is a data tag, u_p(0) Is the middle value of the p-th participating end, and N is the total number of participating ends.

In this embodiment, the model update parameters are calculated and stored by the tag holding end, so when the current model update end is the tag holding end and the current model update end is the feature holding end, the modes of obtaining the model update parameters are different.

In one embodiment, the step of determining the encryption gradient value according to the model update parameter includes: and the current model updating end determines an encryption gradient value according to the local data and the model updating parameters. When the current model updating end is the label holding end, the model updating parameters stored by the current end can be directly obtained, and the model updating parameters and the current model updating end are used for updating the modelCalculates the encrypted gradient values from the local data (i.e., training data). Illustratively, can be according toCalculating an encrypted gradient value, wherein [ [ g ]^p(t)]]For encrypting the gradient values, [ [ d ]_i(t-1)]]The parameters are updated for the model in such a way that,is the local training data of the current model update side.

In one embodiment, the method for determining the encryption gradient value includes that a current model update end is a feature holding end, the current model update end obtains a model update parameter, determines the encryption gradient value according to the model update parameter, and sends the encryption gradient value to a cooperation end, and includes: and the current model updating end acquires the model updating parameters from the label holding end and determines the encryption gradient value according to the local data and the model updating parameters. When the current model update end is the feature holding end, the model update parameters need to be acquired from the tag holding end, and then the encryption gradient value is calculated according to the model update parameters and the local data (i.e. training data) of the current model update end. Also, can be based onCalculating an encrypted gradient value, wherein [ [ g ]^p(t)]]For encrypting the gradient values, [ [ d ]_i(t-1)]]Updating parameters, x, for the model_i ^pIs the local training data of the current model update side.

S130, the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal.

And after receiving the encryption gradient value sent by the current model updating end, the cooperative end determines a model updating direction according to the encryption gradient value and returns the model updating direction to the current model updating end. In the embodiment, in the one-time iteration process, only the current model updating end needs to send the encryption gradient value to the cooperation end, so that data transmission is greatly reduced, and the federal learning efficiency is improved.

In the inventionIn one embodiment, the determining, by the cooperative side, a model update direction according to the encrypted gradient value includes: and the cooperative terminal decrypts the encrypted gradient value to obtain an original gradient value, and determines a model updating direction according to the original gradient value and a preset step length. Specifically, the cooperation terminal decrypts the encrypted gradient value according to a private key to obtain an original gradient value, and then calculates the model updating direction according to the original gradient value and a preset step length. Illustratively, the product of the original gradient value and the preset step size can be used as the model updating direction, i.e. byAnd calculating the updating direction of the model. Wherein the content of the first and second substances,for model update direction, η is the preset step length, g^p(t) is the original gradient value.

And S140, the current model updating end updates the local model parameters according to the model updating direction.

And after the current model updating end receives the model updating direction sent by the cooperation end, updating the local model parameters based on the model updating direction to obtain an updated local model. The local model parameters are updated based on the model updating direction, which refers to the model updating mode in the prior art.

In one embodiment, the current model update terminal updates the local model parameters according to the model update direction, including; and the current model updating end takes the difference value between the current local model parameter and the model updating direction as the updated local model parameter. I.e. can pass throughFor parameters w in the local model^pThe update is performed, wherein,the direction is updated for the model.

The federal learning system provided by the embodiment of the invention comprises: the system comprises a collaboration end and at least two participation ends, wherein each participation end comprises a label holding end and at least one characteristic holding end, the collaboration end is in communication connection with each participation end, and the training of the federal learning system comprises: determining a participating end as a current model updating end; the current model updating end obtains model updating parameters, determines an encryption gradient value according to the model updating parameters, and sends the encryption gradient value to the cooperation end, wherein the model updating parameters are determined by the label holding end according to local model parameters of the last current model updating end; the cooperation terminal determines a model updating direction according to the encryption gradient value and sends the model updating direction to the current model updating terminal; the current model updating end updates the local model parameters according to the model updating direction, and through updating the local models of the participating ends one by one, the participating ends are prevented from simultaneously transmitting a large amount of data to the cooperation end, and the federal learning efficiency is improved.

Optionally, on the basis of the above scheme, after the current model update end updates the local model parameter, the method further includes: and the current model updating end determines an intermediate value according to the updated local model parameters, so that the label holding end updates the model updating parameters according to the intermediate value. In this embodiment, in order to effectively use the information of each iteration, after the local model parameters of the current model update end are updated, the intermediate value is determined according to the updated local model parameters, and the model update parameters in the tag holding end are adjusted by using the intermediate value, so that the intermediate value of the current model update end can cause corresponding influence on the gradient of the whole federal learning system.

For example, the determination of the intermediate value by the current model update end according to the updated local model parameter may be: by u_p(t)＝w^p(t)^Tx^pAn intermediate value is calculated. Where up (t) is an intermediate value, w^p(t) local model parameters, x, for the current model update side^pAnd local training data of the current model updating end.

In an embodiment of the present invention, the updating the model update parameter by the tag holding end according to the intermediate value may be: by passingAnd updating the model updating parameters. The specific meaning of the parameters in the above formula can refer to the above embodiments, and is not described herein again.

In this embodiment, the model update parameters are calculated and stored by the tag holding end, so that when the current model update end is the tag holding end and the current model update end is the feature holding end, the updating modes of the model update parameters are different after the intermediate value is calculated.

In one embodiment, the step of updating the model update parameter by the tag holding end includes: and the current model updating end determines an intermediate value according to the updated local model parameters and updates the model updating parameters according to the intermediate value. Optionally, when the current model update end is the tag holding end, the current model update end may calculate the intermediate value and update the model update parameter at the local end. For example, when the current model update end is the label holding end, the current model update end can pass u_p(t)＝w^p(t)^Tx^pCalculating an intermediate value and then directly based on the calculated intermediate value byAnd updating the model updating parameters.

In one embodiment, the method for updating the model update parameter by the tag holder includes that the current model update terminal is a feature holding terminal, and the current model update terminal determines an intermediate value according to the updated local model parameter, so that the tag holder updates the model update parameter according to the intermediate value, and includes: the current model updating end determines an intermediate value according to the updated local model parameters and sends the intermediate value to the label holding end; and the label holding end updates the model updating parameters according to the intermediate value. Optionally, when the current model update end is the feature holding end, the current model update end can only calculate the intermediate value at the local end, and after the intermediate value is obtained through calculation, the intermediate value is sent to the label holding end, and the label holding end receives the intermediate valueAnd updating the model updating parameters according to the intermediate value. For example, when the current model update end is the feature holding end, the current model update end can pass u_p(t)＝w^p(t)^Tx^pCalculating the intermediate value, sending the intermediate value to the tag holding end, and passing the intermediate value through the tag holding endAnd updating the model updating parameters.

In general, no matter whether the current model update end is the tag holding end or the feature holding end, the intermediate value needs to be recalculated after the local model parameter of the current model update end is updated, and then the tag holding end updates the model update parameter according to the intermediate value calculated after the current model update end is updated. And the updated model updating parameters are used for updating the next model updating end, the updating logic of the next model updating end can refer to the updating logic of the current model updating end until the local models of all the participating ends are converged, and the iteration is stopped to obtain the trained federated learning system. It is to be understood that, the manner of determining the convergence of the model may refer to the manner of determining the convergence of the model in the prior art, and is not limited herein.

According to the embodiment, in one iteration, the information encrypted by the last iteration is effectively utilized, high calculation complexity and communication complexity caused by calculation and transmission of information of all participating terminals every time are avoided, and model updating is realized by only updating one party of information. Meanwhile, although only one model is updated, the updated intermediate value affects the overall gradient and is propagated to the next participating end in the gradient calculation of the next iteration. In the process, if the label holding end is updated, the label holding end and the cooperation end are only required to communicate back and forth once; if the feature holding end is updated, only one feature holding end needs to communicate back and forth with the cooperation end once and only one feature holding end needs to communicate back and forth with the label holding end once. Therefore, the scheme provided by the embodiment of the invention greatly reduces the data transmission quantity of each iteration. The communication overhead per iteration of the embodiment of the invention can be compared with the prior artThe protocols were compared. The specific comparison is as follows: assuming that the total number of data samples is N, the characteristic dimension of the p-th participating end (i.e. the current model updating end) is m^pEach value is F bits before encryption and F bits after encryption_eA bit. The communication overhead per iteration of the existing scheme isA bit. For the embodiment of the invention: if p is the Host side: the communication cost of each iteration mainly comes from model updating parameter transmission from the tag holding end to the current model updating end, encryption gradient value transmission from the current model updating end to the cooperation end, model updating direction transmission from the cooperation end to the current model updating end and intermediate value transmission from the current model updating end to the tag holding end, and the communication cost is NF respectively_e、m^pF_e、NF、NF_eThen the total overhead is 2NF_e+m^p(F+F_e) A bit; if p is Guest party: the communication overhead of each iteration is mainly from the transmission of the encryption gradient value from the current model updating end to the cooperation end and the transmission of the model updating direction from the cooperation end to the current model updating end, and the communication overhead is m^pF_e、m^pF, then the total overhead is m^p(F+F_e) A bit. Therefore, compared with the existing scheme, the method and the device can greatly reduce the transmission quantity of each iteration, thereby improving the efficiency of federal learning.

In one embodiment of the present invention, the method further comprises: and the cooperative end judges whether a convergence condition is reached according to the encryption gradient values sent by the participating ends, and judges that the federated learning system converges when the convergence condition is reached. Namely, the cooperation terminal judges whether the local model of each participating terminal reaches the convergence condition according to the encryption gradient value sent by each participating terminal after iteration, and when the local model of each participating terminal reaches the convergence condition, the federal learning system is judged to be converged. Whether the local model of each participating end is converged is judged through the cooperation end so as to judge whether the federal learning system is converged, information received by the cooperation end is fully utilized, and the efficiency of federal learning is improved.

Optionally, the cooperative side sends according to each participating sideThe judgment of whether the convergence condition is reached by the encryption gradient value comprises the following steps: aiming at the encryption gradient value sent by each participating end, the cooperation end judges whether the characteristic value of the encryption gradient value is smaller than a set threshold value or not; and when the characteristic values of the encryption gradient values of all the participating ends are smaller than the set threshold value, judging that the convergence condition is reached. For example, after receiving the encrypted gradient value sent by any participating end, the cooperative end may decrypt the encrypted gradient value using a private key to obtain an original gradient value, calculate a feature value of the original gradient value, and compare the feature value of the original gradient value with a preset threshold to determine whether the local model of the participating end converges. The characteristic value of the original gradient value can be any characteristic value of the original gradient value, and the specific setting can be determined according to actual requirements. If the characteristic value of the original gradient value can be a two-norm of the original gradient value, the corresponding preset threshold can be set to 10^-3。

Example two

The present embodiment provides a preferred embodiment based on the above-described embodiments. In this embodiment, the tag holding end is embodied as Guest, the feature holding end is embodied as Host, and the cooperative end is embodied as third party Coordinator.

In this embodiment, it is assumed that there are P participating peers, one participating peer is Guest, the other P-1 participating peers are Host, and a third party Coordinator is provided to coordinate the whole process. Recording the local data of the p-th participant end after the private data alignment as x^pThe model parameter is w^p. Before iterative training, each participant needs to be initialized. Specifically, each participating end generates an initial model parameter w^p(0) And then, respectively locally calculating intermediate values, and transmitting the ciphertext to the Guest party by each Host party through addition homomorphic encryption (such as Paillier). After the Guest party obtains all the intermediate values, the model updating parameters are calculated according to the principle of addition homomorphic encryption. And after the initialization is finished, entering an iterative training stage.

Fig. 2 is a schematic flowchart of a single iterative training of a bang learning system according to a second embodiment of the present invention. As shown in fig. 2, fig. 2 schematically shows a training process of the tth iteration, which includes the following specific steps:

step 1: selecting a p-th participating end as a model updating party (namely a current model updating end) of the iteration;

step 2: if the p-th participating end is the Host party, the Guest party transmits the model updating parameters to p, otherwise, Step3 is directly executed;

step 3: the p-th participant calculates an encryption gradient value according to the local data and transmits the encryption gradient value to the Coordinator;

step 4: the Coordinator decrypts the gradient, multiplies the step length to calculate the model updating direction, and returns the model updating direction to the pth participating end;

step 5: the p-th participating end updates the local model parameters, and the updating formula can be

Step 6: the participating terminal p recalculates the intermediate value according to the updated local model parameter;

specifically, if the p-th participating end is a Host party, the intermediate value is encrypted and then transmitted to Guest, and if the p-th participating end is a Guest party, the intermediate value is directly calculated locally;

step 7: updating the model updating parameters by the Guest party according to the intermediate value of p;

the method specifically comprises the following steps:

step 8: t ← t +1, go to the next iteration.

Step1 requires selecting a participating peer p, and the specific selection method can refer to the above embodiments. During training, the above process is executed iteratively until the algorithm converges, and whether the algorithm converges or not can be judged by using a gradient: for example, if the Coordinator finds that the two-norm gradient values of each side are less than a set threshold (e.g., 10)^-3) Then the algorithm is considered to be converged and the iteration is stopped.

The embodiment of the invention provides a longitudinal federal learning framework of a multi-party participated co-constructed logistic regression model, and provides a model updating scheme with high transmission efficiency, wherein only at most one host party is required to participate in each iteration process, so that only a small number of participating terminals are required to update model parameters in each iteration, the times of mutual transmission of data in each iteration are reduced, and the overall communication efficiency is improved; and an addition homomorphic encryption scheme is used, so that the safety of intermediate results is protected, and the data privacy is prevented from being disclosed to other parties.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments illustrated herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.