Control method, control management platform, device, medium, and program product for neural network model

文档序号：137264 发布日期：2021-10-22 浏览：16次中文

阅读说明：本技术 神经网络模型的控制方法、控制管理平台、设备、介质及程序产品 (Control method, control management platform, device, medium, and program product for neural network model ) 是由孙磊戴乐育毛秀青王粤晗郭松郭松辉胡翠云李作辉窦睿彧于 2021-07-06 设计创作，主要内容包括：本申请提供了一种神经网络模型的控制方法,该方法应用于控制管理平台,接收来自用户访问该神经网络模型的访问请求,当该用户获得授权时,获取与该用户的访问请求对应的控制参数的控制信息,其中控制参数包括该神经网络模型的权重参数、偏置参数和激活参数中的至少一个,然后根据控制参数的控制信息对该神经网络模型进行控制。如此,避免非授权用户通过神经网络模型的输出结果,推断神经网络模型,进而攻击该神经网络模型,从而提高了神经网络模型的安全性,保护了用户的隐私数据。(The application provides a control method of a neural network model, which is applied to a control management platform, receives an access request from a user for accessing the neural network model, acquires control information of control parameters corresponding to the access request of the user when the user obtains authorization, wherein the control parameters comprise at least one of weight parameters, bias parameters and activation parameters of the neural network model, and then controls the neural network model according to the control information of the control parameters. Therefore, the method avoids that an unauthorized user infers the neural network model through the output result of the neural network model and further attacks the neural network model, thereby improving the safety of the neural network model and protecting the privacy data of the user.)

1. A control method of a neural network model is applied to a control management platform, and the method comprises the following steps:

receiving an access request from a user, the access request for accessing the neural network model;

when the user obtains authorization, obtaining control information of control parameters corresponding to the access request of the user, wherein the control parameters comprise at least one of weight parameters, bias parameters and activation functions of the neural network model;

and controlling the neural network model according to the control information of the control parameters.

2. The method of claim 1, wherein receiving an access request from a user comprises:

receiving a first access request and a second access request from the user;

the control information of the control parameter corresponding to the first access request is different from the control information of the control parameter corresponding to the second access request.

3. The method of claim 1, wherein the control information comprises at least one of an evolution factor of the control parameter and location information of the control parameter, the location information of the control parameter being used to characterize a location of the control parameter in the neural network model.

4. The method of claim 3, further comprising:

generating a random number for the user;

the position information of the control parameter is determined by the random number.

5. The method of claim 4, wherein the random number is a binary random number of times;

the method further comprises the following steps:

and determining the number of control parameters in the neural network model through a sliding window and the binary random number.

6. The method according to any one of claims 1 to 5, wherein the controlling the neural network model according to the control information of the control parameter comprises:

generating an authorization operator according to the control information of the control parameter;

returning the authorization operator to the user;

receiving input data from the user, the input data comprising data to be processed and the authorization operator;

obtaining predictive data of the input data through the neural network model under control;

returning predictive data of the input data to the user.

7. The method of any one of claims 1 to 5, wherein the neural network model comprises at least one of a convolutional neural network model, a cyclic neural network model, or a recursive neural network model.

8. A control management platform of a neural network model, the control management platform comprising:

an access request receiving module, configured to receive an access request from a user, where the access request is used to access the neural network model;

a control information obtaining module, configured to obtain control information of a control parameter corresponding to an access request of the user when the user obtains authorization, where the control parameter includes at least one of a weight parameter, a bias parameter, and an activation function of the neural network model;

and the neural network model control module is used for controlling the neural network model according to the control information of the control parameters.

9. An apparatus, comprising a processor and a memory;

the processor is to execute instructions stored in the memory to cause the device to perform the method of any of claims 1 to 7.

10. A computer-readable storage medium comprising instructions that direct a device to perform the method of any of claims 1-7.

Technical Field

The present application relates to the field of Artificial Intelligence (AI), and in particular, to a control method, a control management platform, a device, a computer-readable storage medium, and a computer program product for a neural network model.

Background

The development of cloud computing (cloud computing) provides computational support for the development and deployment of AI applications, and therefore, more and more developers choose to train a neural network model by using a cloud computing platform (i.e., a cloud platform) so as to construct an AI application and deploy the AI application on the cloud platform.

In the cloud service mode, a developer or operator of the AI application may deploy an AI reference constructed by a trained neural network model, e.g., a deep neural network model such as a convolutional neural network model, a cyclic neural network model, etc., to the cloud platform for use by authorized cloud users.

However, due to vulnerability in system level protection of the cloud platform, unauthorized users may illegally use the neural network model by bypassing defense mechanisms. For example, an unauthorized user can access the neural network model in the cloud platform for multiple times, adopt a violent means, such as equation solving attack, and copy a model identical to the original neural network model. The unauthorized user can use a plagiarism model to predict data, or carry out black box attack and gray box attack on a neural network model in a cloud platform to cause prediction errors, or steal user privacy data, for example, different drug lists are given according to the diseases of different patients in an intelligent medicine dispensing system. Thus, serious safety hazards can result.

There is a need in the art to provide a method for access control of a neural network model in AI applications to reduce the potential safety hazard.

Disclosure of Invention

The application provides a control method of a neural network model, which can enable an unauthorized user to access the neural network model to obtain an abnormal result, thereby improving the safety of the neural network model. The application also provides a control management platform, equipment, a computer readable storage medium and a computer program product corresponding to the method.

In a first aspect, the present application provides a method for controlling a neural network model. The method is applied to a control management platform and comprises the following steps:

receiving an access request from a user, wherein the access request is used for accessing the neural network model;

when a user obtains authorization, obtaining control information of control parameters corresponding to an access request of the user, wherein the control parameters comprise at least one of weight parameters, bias parameters and activation functions of a neural network model;

and controlling the neural network model according to the control information of the control parameters.

In some possible implementations, receiving an access request from a user includes:

receiving a first access request and a second access request from a user;

In some possible implementations, the control information includes at least one of an evolution factor of the control parameter and location information of the control parameter, the location information of the control parameter being used to characterize a location of the control parameter in the neural network model.

In some possible implementations, the method further includes:

generating a random number for a user;

the position information of the control parameter is determined by a random number.

In some possible implementations, the random number is a binary random number of times;

the method further comprises the following steps:

and determining the number of control parameters in the neural network model through a sliding window and a binary random number.

In some possible implementations, the controlling the neural network model according to the control information of the control parameter includes:

generating an authorization operator according to the control information of the control parameter;

returning an authorization operator to the user;

receiving input data from a user, wherein the input data comprises data to be processed and an authorization operator;

obtaining the prediction data of the input data through a controlled neural network model;

the predicted data for the input data is returned to the user.

In some possible implementations, the neural network model includes at least one of a convolutional neural network model, a recurrent neural network model, or a recurrent neural network model.

In a second aspect, the present application provides a control management platform of a neural network model, the control management platform comprising:

the access request receiving module is used for receiving an access request from a user, and the access request is used for accessing the neural network model;

the control information acquisition module is used for acquiring control information of control parameters corresponding to the access request of the user when the user obtains authorization, wherein the control parameters comprise at least one of weight parameters, bias parameters and activation functions of the neural network model;

and the neural network model control module is used for controlling the neural network model according to the control information of the control parameters.

In some possible implementations, the access request receiving module is specifically configured to:

receiving a first access request and a second access request from a user;

In some possible implementations, the control management platform further includes a random number generation module:

the random number generation module is used for generating a random number aiming at a user;

the position information of the control parameter is determined by a random number.

In some possible implementations, the random number is a binary random number of times;

the control management platform also comprises a control parameter number determining module:

and the control parameter number determining module is used for determining the number of control parameters in the neural network model through the sliding window and the binary random number.

In some possible implementations, the neural network model control module is specifically configured to:

generating an authorization operator according to the control information of the control parameter;

returning an authorization operator to the user;

receiving input data from a user, wherein the input data comprises data to be processed and an authorization operator;

obtaining the prediction data of the input data through a controlled neural network model;

the predicted data for the input data is returned to the user.

In some possible implementations, the neural network model includes at least one of a convolutional neural network model, a recurrent neural network model, or a recurrent neural network model.

In a third aspect, the present application provides an apparatus comprising a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory to cause the apparatus to perform a method of controlling a neural network model as in the first aspect or any implementation of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, the instructions instructing a device to execute a method for controlling a neural network model according to the first aspect or any implementation manner of the first aspect.

In a fifth aspect, the present application provides a computer program product comprising instructions which, when run on a device, cause the device to perform the method of controlling a neural network model according to the first aspect or any one of the implementations of the first aspect.

The present application can further combine to provide more implementations on the basis of the implementations provided by the above aspects.

According to the technical scheme, the embodiment of the application has the following advantages:

the application provides a control method of a neural network model, which is applied to a control management platform and specifically comprises the steps of receiving an access request from a user for accessing the neural network model, acquiring control information of control parameters corresponding to the access request of the user when the user obtains authorization, wherein the control parameters comprise at least one of weight parameters, bias parameters and activation parameters of the neural network model, and then controlling the neural network model according to the control information of the control parameters. Therefore, the method avoids the unauthorized user from deducing the neural network model through the output result of the neural network model so as to attack the neural network model, improves the safety of the neural network model and protects the privacy data of the user.

Drawings

In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings needed to be used in the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without inventive labor.

FIG. 1 is a schematic diagram of a neural network model for theft of an adversary according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of an inactive state and an active state of an activation function according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a control method of a neural network model according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of a control method of a neural network model according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram illustrating control of a plurality of parameters in a convolutional neural network model according to an embodiment of the present disclosure;

FIG. 6 is a diagram illustrating location markers of intrinsic parameter points of a neural network model according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart of a process for controlling a management platform to generate parameters of an authorized user information table according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a control management platform of a neural network model according to an embodiment of the present application.

Detailed Description

The scheme in the embodiments provided in the present application will be described below with reference to the drawings in the present application.

The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

Some technical terms referred to in the embodiments of the present application will be first described.

Cloud computing is one of distributed computing, and means that huge data computing processing programs are decomposed into countless small programs through a network cloud, and then the small programs are processed and analyzed through a system consisting of a plurality of servers to obtain results and the results are returned to users. By this technique, tens of thousands of data can be processed in a short time (for example, several seconds), thereby achieving a strong network service.

The development of cloud computing can provide computing support for artificial intelligence, and developers of neural network models can deploy the neural network models on a cloud platform, so that users can use the trained neural network only by calling Application Programming Interfaces (APIs) through the cloud platform.

There are many security issues with such artificial intelligence services. On one hand, when the user uses the own data to predict on the cloud platform, the own data of the user may include the privacy of the user, and therefore, the stealing of the data may mean that the privacy of the user is stolen. On the other hand, the system level protection of the cloud platform is vulnerable, and the model itself may also encounter security problems such as parameter stealing and unauthorized user access.

Many scholars at home and abroad consider protecting the privacy of users and propose methods such as multi-party calculation, but few scholars pay attention to the vulnerability of the neural network model.

Specifically, as shown in fig. 1, an unauthorized user (adversary) can bypass a defense mechanism to illegally use a convolutional neural network model, and hack a adversary model identical to the original convolutional neural network model by accessing the neural network multiple times and using a violent means, such as equation solving attack. The adversary can freely use the plagiarism convolutional neural network model to predict data, and can also use the convolutional neural network to launch black box attacks and gray box attacks on the original cloud convolutional neural network, so that the convolutional neural network prediction is wrong, or the private data of the user is stolen.

The black box attack means that the structure of the original neural network model cannot be obtained, but the same data as the original neural network model can be found as far as possible, and a new neural network model is trained by using the data.

Generally, the convolutional neural network model is more widely applied than other deep neural network models, and therefore the convolutional neural network model is mainly used as an introduction object in the present embodiment for introduction, but the neural network model in the present application may also be a cyclic neural network model or a recurrent neural network model, and the present application is not limited herein.

In the convolutional neural network algorithm, the main function of an activation function is to perform nonlinear operation on data obtained by performing convolutional operation on feature data and weights, and common activation functions include ReLU, sigmod, tanh and the like. All activation functions can be divided into two states according to the result of the output data: the activated state and the deactivated state, as shown in fig. 2. In the graph, the circle part region is input with different data by the activation function, but the output result is not very different, that is, in the region, the activation function cannot distinguish the difference of the input data, the activation function is in a non-working state, and therefore the region is called an inactive state, the input data region is called an inactive state x region, and the output data region is called an inactive state y region.

Adding an activity control function of the activation function at an input end of the activation function, namely inputting an activity control key before the neural network operation, so that the data is firstly subjected to the operation of the activity control function through the activity control key before the activation function operation is carried out, and if the activity control key is incorrect, the activity control function changes a data value, so that an operation result of the activity control function is in an inactive state area of the activation function, and the subsequent neural network operation cannot be normally carried out; if the activity control key is correct, the activity control function keeps the original data value, and the subsequent neural network calculation can be normally carried out.

The activity control function refers to a function in which an activity control key and characteristic data participate in operation together before the operation of the activation function, so as to control the activity of the activation function of the convolutional neural network. The activity control function according to the present embodiment may include a zoom-in activity control function, a zoom-out activity control function, an inverse activity control function, a judgment selection activity control function, a random activity control function, and the like.

The position point of the activity control function refers to the position of the activity control function participating in the operation process of the convolutional neural network. The position points of the activity control function related to the scheme comprise the weight, the offset and the activation function operation of the convolutional neural network.

In view of the above, the present application provides a method for controlling a neural network model in an AI application. The method comprises a multi-control neuron, namely a plurality of controlled neurons can be included in a convolutional neural network model, wherein the neurons can be in the same convolutional layer or different convolutional layers, and the same control function or different control functions can be used. Specifically, the activity control function may be set at different position points of the convolutional neural network, so that the activities of a plurality of neurons of the convolutional neural network are controlled, and the activity control function performs activity operation on the activity control key and the operation data, thereby implementing control of the convolutional neural network.

Further, since the activity control key is a key for controlling the whole convolutional neural network, and once the activity control function and the position point in the activity control function are determined, the activity control key is fixed, in order to protect the activity control key, the random number seed may be added, so that the random number seed of the activity control key used by the user each time is different, and the change is realized once.

The control method of the neural network model provided by the embodiment is applied to a control management platform, and specifically includes receiving an access request from a user for accessing the neural network model, acquiring control information of a control parameter corresponding to the access request of the user when the user obtains authorization, wherein the control parameter includes at least one of a weight parameter, a bias parameter and an activation parameter of the neural network model, and then controlling the neural network model according to the control information of the control parameter.

The control management platform may be software, and the software may be deployed in a computer cluster in a centralized manner or in a distributed manner. The control management platform may also be hardware that is used to control the neural network model.

The method may be performed by a user side, a cloud administrator side, a control management platform, and a neural network application server together, as shown in fig. 3, where the user side may be a Personal Computer (PC), and the control management platform and the neural network application server may use two servers respectively.

The specific work flow comprises the following steps: and uploading the trained neural network model to a neural network application server from a background through a secure channel. The user to be authorized establishes communication with the control management platform through the secure channel, and the control management platform receives an access request of the user and forwards the access request to the cloud administrator. After the cloud administrator agrees and authorizes the user, the control management platform obtains control information of control parameters corresponding to the user access request, and controls a plurality of parameters such as weight parameters, bias parameters, activation functions and the like of the neural network, as shown in fig. 3.

For convenience of understanding, the control method of the neural network model provided by the embodiment of the present application is described below with reference to the drawings.

Referring to a flow chart of a control method of the neural network model shown in fig. 4, specific steps of the method are as follows.

S402: the control management platform receives an access request from a user.

Wherein the access request is for accessing the neural network model. In some possible implementations, the neural network model may be any one of a convolutional neural network model, a recurrent neural network model, or a recurrent neural network model.

Optionally, the control management platform receives a first access request and a second access request from a user, where control information of a control parameter corresponding to the first access request is different from control information of a control parameter corresponding to the second access request, and the user may access the control management platform multiple times through multiple access requests.

S404: when the user obtains authorization, the control management platform obtains the control information of the control parameters corresponding to the access request of the user.

Wherein the control parameters include at least one of a weight parameter, a bias parameter, and an activation function of the neural network model.

Specifically, when the user obtains authorization, the control management platform obtains at least one of a weight parameter, a bias parameter, or a control parameter such as an activation function in the neural network model corresponding to the access request of the user, and the evolution factor and the position information corresponding to the weight parameter, the bias parameter, or the activation function, respectively. Wherein the position information of the control parameter can be used to characterize the position of the control parameter in the neural network model.

Taking an activation function as an example, each convolutional layer of the convolutional neural network comprises a plurality of neurons, the activation functions of the convolutional neural network are modified, an activity control mechanism and an activity control parameter can be added to the activation function of each neuron, when the activity control parameter is correct, the activation function is activated, and the convolutional neural network can work normally; when the activity control parameter is incorrect, the activation function cannot be activated, and the convolutional neural network cannot work normally.

Because a plurality of neurons usually adopt the same and fixed activity control parameters, the safety of the activity control parameters and the activity control mechanism is difficult to guarantee, and the activity control parameters and the activity control mechanism can still be easily attacked and identified by an enemy, so that the activity control parameters special for each neuron can be set, and the parameters are combined to form an activity control matrix of the convolutional neural network.

The value of the activity control matrix of the convolutional neural network can be realized by adopting various methods, such as a randomness value taking method, a password encryption protection method and an authorization dynamic change method, so that the security of authorization control is improved.

In this embodiment, the control method of the neural network model needs to control multiple parameters such as a weight parameter, a bias parameter, and an activation function in the neural network, as shown in fig. 5. Therefore, the control management platform stores a control list, and the control list includes authorized user information, random number seeds r corresponding to each user, evolution factors k generated by the seeds, position information l of the evolution factors, and the like. In some possible implementations, the value r of the random seed, the evolution factor k and the location information l thereof are dynamically changed automatically every time the controlled neural network is used by the authorized user, as shown in table 1.

Table 1 control management platform authorized user information table

The control management platform can arrange parameter points such as weight parameters, bias parameters and activation functions of the neural network in sequence according to the neural network structure, the first hidden layer weight w1 parameter point is marked as 1, the first hidden layer bias b1 parameter point is marked as 2, the first hidden layer activation function g1 point is marked as 3, the process is repeated in … …, and the schematic diagram of the position mark of the parameter points in the neural network can be as shown in fig. 6.

In some possible implementation manners, the access request of the user includes identity information of the user, and the administrator authorizes the user meeting the authorization requirement according to the identity information of the user.

And the control management platform generates a random number by using the identity information of the authorized user according to the dynamic control strategy and generates an evolution factor and position information of the control parameter by using the random number. The identity information of the authorized user may include a user Identity Document (ID), and the user IDs of different users are different.

And when the random number is a binary random number, determining the number of the control parameters in the neural network model through a sliding window and the binary random number.

In particular, the control management platform may generate different binary random numbers r for different users_OThe generation of the random number may be facilitated by hardware, such as a random number generator.

For the deep convolutional neural network with the total number of the parameter points being n, the control management platform generates the bit number ofSliding window of (1), from right to left to random number r_OTaking value until the random number num is taken_{(user id)}Less than the total number n of the parameter points to obtain the number num of the parameter points to be controlled_{(user id)}. Continuing to carry out sliding value taking on the source random number, and recording the value taken as position information l_iUntil i is num_{(user id)}And when the information is full, controlling the management platform to generate a flow chart of the parameters of the authorized user information table, as shown in fig. 7.

After a cloud administrator authorizes a user, a control management platform generates position point information l, and an evolution factor K is generated at a corresponding parameter point by using a control function and taking a random number as a seed, wherein K is { K ═ K }_w，k_b，k_g}。

In some possible implementation modes, a user uses the controlled convolutional neural network every time, updates the random number of seeds, and further generates a new evolution factor group and position information, so that the convolutional neural network is dynamically controlled in real time.

S406: and the control management platform controls the neural network model according to the control information of the control parameters.

Specifically, the control management platform generates an authorization operator according to control information of the control parameter, then returns the authorization operator to a user sending an access request, then receives input data from the user, wherein the input data comprises data to be processed and the authorization operator, acquires prediction data of the input data through a controlled neural network model, and then returns the prediction data of the input data to the user.

And the control management platform calculates an authorization operator according to the value of the evolution factor and distributes the authorization operator to authorized users.

After the authorized user obtains the authorized information, the data processing can be carried out by using the neural network. Specifically, the user sends the data to be processed and the authorization operator to the control management platform, and the data to be processed and the authorization operator are input data which are jointly used as the control management platform. And after receiving the input data, the control management platform detects whether the input data contains an authorization operator, and inputs the authorization operator and the data to be processed into the controlled convolutional neural network together. The convolutional neural network decouples the authorization operator and the evolution factor by using the parameter position information corresponding to the user, processes the input data and obtains the prediction data as output.

For example, authorized users can use a controlled convolutional neural network model for image classification. Specifically, a user can input a batch of fruit pictures and an authorization operator, the control management platform detects that the authorization operator and the pictures contain the authorization operator, then the authorization operator and the pictures are input into the controlled convolutional neural network model together, the controlled convolutional neural network model decouples the authorization operator and the evolution factor by using parameter position information corresponding to the user, then the fruit pictures to be processed are input into the convolutional neural network model for classification, and therefore the fruit pictures which are correctly classified are output.

Further, after obtaining feedback of the completion of the use of the convolutional neural network, the control management platform generates a new group of random numbers according to the user ID, updates the parameter evolution factor and the position information of the neural network model according to the new random numbers, and calculates the authorization operator according to the value of the evolution factor. After each access is finished, the user can automatically obtain the authorization operator of the next access, so that the control access to the neural network model is realized.

And the control management platform returns the new authorization operator and the output data to the user side together, so that the authorized user can obtain correct classified output. Therefore, the control management platform can identify the user through the authorization operator, so that the unauthorized user is prevented from using the neural network model.

When the control management platform receives data to be processed input by an unauthorized user, whether the input data contains an authorized operator is detected, the control management platform further inputs the data to be processed without the authorized operator into the controlled neural network model, the controlled neural network model outputs the data to be processed after recognizing the data to be processed as a tool kit because the authorized operator is not detected, the unauthorized user obtains error output, and the structure of the neural network model cannot be estimated according to a large amount of error output, so that the neural network model can be protected.

Therefore, the control method of the neural network model embeds a new authorization control mechanism in the neural network model, when an unauthorized user uses the neural network model, the unauthorized user can not normally use the neural network model because the interior of the model is controlled by the authorization control mechanism, and the unauthorized user obtains abnormal error output; and the authorized user can carry the control information of the control parameter to remove the control of the authorization control mechanism on the neural network model, thereby obtaining correct output. The method can avoid that an illegal user deduces the neural network model through the output result of the neural network model so as to attack the neural network model. Based on the method, the output result obtained by the unauthorized user is wrong and abnormal, so that the structure of the neural network model cannot be deduced, the safety of the neural network model is ensured, and the privacy data of the user is protected.

Corresponding to the above method embodiment, the present application also provides a control management platform of a neural network model, referring to fig. 8, where the control management platform 800 includes: an access request receiving module 802, a control information obtaining module 804, and a neural network model control module 806.

The access request receiving module is used for receiving an access request from a user, and the access request is used for accessing the neural network model;