阅读说明：本技术 一种确定安全级别的方法及装置、存储介质和设备 (Method and device for determining security level, storage medium and equipment ) 是由 纪翔 饶培伦 于 2019-09-23 设计创作，主要内容包括：本申请实施例提供一种确定安全级别的方法及装置、存储介质和设备,所述方法包括：将预先设置的问题库中的问题发送给待分级的人工智能产品；获取所述人工智能产品输出的所述问题的回复信息；基于所述回复信息确定所述人工智能产品的安全级别。如此,能够提高人工智能产品的安全级别准确度,并能够有效避免确定安全级别过程中的标准不一致的问题。(The embodiment of the application provides a method and a device for determining a security level, a storage medium and equipment, wherein the method comprises the following steps: sending the problems in the preset problem library to the artificial intelligence product to be classified; acquiring reply information of the question output by the artificial intelligence product; determining a security level of the artificial intelligence product based on the reply information. Therefore, the safety level accuracy of the artificial intelligence product can be improved, and the problem of inconsistent standards in the safety level determining process can be effectively avoided.)

1. A method of determining a security level, the method comprising:

sending the problems in the preset problem library to the artificial intelligence product to be classified;

acquiring reply information of the question output by the artificial intelligence product;

determining a security level of the artificial intelligence product based on the reply information.

2. The method of claim 1, wherein determining the security level of the artificial intelligence product based on the reply information comprises:

performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result;

determining a security level of the artificial intelligence product based at least on the text analysis results.

3. The method of claim 2, wherein determining the security level of the artificial intelligence product based at least on the text analysis results comprises:

performing semantic understanding on the reply information to obtain a semantic understanding result;

determining a security level of the artificial intelligence product based on the semantic understanding result and the text analysis result.

4. The method of claim 3, wherein determining the security level of the artificial intelligence product based on the semantic understanding result and the text analysis result comprises:

determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result;

determining a second security level corresponding to the artificial intelligence product based on the text analysis result;

determining a security level of the artificial intelligence product based on the first security level and the second security level.

5. The method of claim 2, wherein the pre-set ranking criteria comprises: and mapping relations between the preset keywords and the corresponding word frequencies and the security levels of the preset keywords.

6. The method according to claim 5, wherein the performing text analysis on the reply message based on a preset grading criterion to obtain a text analysis result comprises: matching the preset keywords with the reply information; determining a target keyword appearing in the reply information in the plurality of preset keywords according to a matching result, and counting the frequency of the target keyword appearing in the reply information; determining the frequency of the target keywords appearing in the reply information as the text analysis result;

the determining a security level of the artificial intelligence product based at least on the text analysis results includes: and determining the safety level of the artificial intelligence product at least based on the frequency of the target keywords appearing in the reply information and the mapping relation between the corresponding word frequency and the safety level.

7. The method of claim 6, wherein determining the security level of the artificial intelligence product based on at least the mapping relationship between the occurrence frequency of the target keyword in the reply message and the corresponding word frequency and security level comprises:

if the number of the target keywords is multiple, determining the security level corresponding to the frequency of each target keyword appearing in the reply information according to the mapping relation between the word frequency and the security level corresponding to each target keyword, and obtaining multiple candidate levels corresponding to the artificial intelligent product;

determining a security level of the artificial intelligence product based at least on a lowest security level of the plurality of candidate levels or a highest number of occurrences of the plurality of candidate levels.

8. The method of claim 1, wherein determining the security level of the artificial intelligence product based on the reply information comprises:

performing semantic understanding on the reply information to obtain a semantic understanding result;

determining a security level of the artificial intelligence product based at least on the semantic understanding result.

9. The method of claim 1, wherein after the determining the security level of the artificial intelligence product based on the reply information, the method further comprises:

displaying the security level of the artificial intelligence product;

responding to the modification operation of the user on the security level, and acquiring the security level input by the user aiming at the artificial intelligence product;

and modifying the safety level of the artificial intelligence product into the safety level input by the user.

10. An apparatus for determining a security level, the apparatus comprising:

the sending module is used for sending the problems in the preset problem library to the artificial intelligence products to be classified;

the acquisition module is used for acquiring reply information of the question output by the artificial intelligence product;

a determination module to determine a security level of the artificial intelligence product based on the reply information.

11. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls a computer device on which the storage medium resides to perform the steps of the method of determining a security level of any one of claims 1 to 9.

12. A computer device, characterized in that the computer device comprises:

at least one processor;

and at least one memory, bus connected with the processor;

the processor and the memory complete mutual communication through the bus; the processor is adapted to invoke program instructions in the memory to perform the steps of the method of determining a security level of any of claims 1 to 9.

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for determining a security level, a storage medium, and a device.

Background

In recent years, with the gradual fall of artificial intelligence technology, more and more artificial intelligence products, such as voice intelligence assistants, intelligent chat robots, intelligent customer service and the like, are widely applied to daily life and work of people.

However, after being released and used, the artificial intelligence product has strong learning ability and memory ability, and can absorb learning information which can bring potential harm to the user from the user language, internet data resources or other interaction processes in the whole life cycle. Therefore, the high intelligent degree of the artificial intelligence product provides convenient and effective service for the user and brings certain potential use risk to the user. For example, in the process of a user interacting with an artificial intelligence product, if the artificial intelligence product often outputs bad information to the user, such as dirty words, biased speeches, popular content, content violating laws, content violating morals, inappropriate hospitalizing advertisements, and the like, the physical and mental health of the user is easily affected negatively.

At present, in order to find out the potential harm of artificial intelligence products in the aspects of society, morality, emotion or health and the like and reduce the potential harm brought by the artificial intelligence products, the safety level of the artificial intelligence products is mainly determined manually by professional graders so that users can know the possible harm brought by different artificial intelligence products and select proper artificial intelligence products. However, the safety level of the artificial intelligence product is determined manually, so that the problem that the accuracy of the safety level of the artificial intelligence product is low and the problem that the standards in the safety level determining process are inconsistent exist.

Disclosure of Invention

In view of this, embodiments of the present application provide a method and an apparatus for determining a security level, a storage medium, and a device, which determine the security level of an artificial intelligence product according to reply information output by the artificial intelligence product to a problem in a preset problem library, can improve accuracy of the security level of the artificial intelligence product, and can effectively avoid a problem of inconsistent standards in a process of determining the security level.

The embodiment of the application mainly provides the following technical scheme:

in a first aspect, an embodiment of the present application provides a method for determining a security level, where the method includes: sending the problems in the preset problem library to the artificial intelligence product to be classified; acquiring reply information of the question output by the artificial intelligence product; determining a security level of the artificial intelligence product based on the reply information.

In a second aspect, an embodiment of the present application provides an apparatus for determining a security level, where the apparatus includes: the sending module is used for sending the problems in the preset problem library to the artificial intelligence products to be classified; the acquisition module is used for acquiring reply information of the question output by the artificial intelligence product; a determination module to determine a security level of the artificial intelligence product based on the reply information.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where the storage medium includes a stored program, where the program, when executed, controls a computer device in which the storage medium is located to perform the above-mentioned steps of the method for determining a security level.

In a fourth aspect, an embodiment of the present application provides a computer device, where the computer device includes: at least one processor; and at least one memory, bus connected with the processor; the processor and the memory complete mutual communication through the bus; the processor is configured to call program instructions in the memory to perform the steps of the method for determining a security level described above.

According to the method, the device, the storage medium and the equipment for determining the security level, after the problems in the preset problem library are sent to the artificial intelligence product to be classified, the reply information aiming at the problems output by the artificial intelligence product can be automatically acquired, then, the artificial intelligence product can be classified based on the reply information, and the security level of the artificial intelligence product is acquired. Therefore, because the reply information of the artificial intelligence product to the problems in the question bank is automatically acquired through the preset question bank, and the safety level of the artificial intelligence product is determined according to the reply information, the accuracy of the safety level of the artificial intelligence product can be improved, and the problem of inconsistent standards in the process of determining the safety level can be effectively avoided.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. Other advantages of the application may be realized and attained by the instrumentalities and combinations particularly pointed out in the specification, claims, and drawings.

Drawings

The accompanying drawings are included to provide an understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the examples serve to explain the principles of the disclosure and not to limit the disclosure.

Fig. 1 is a first flowchart illustrating a method for determining a security level in an embodiment of the present application;

fig. 2 is a flowchart illustrating a second method for determining a security level in an embodiment of the present application;

fig. 3 is a third flowchart illustrating a method for determining a security level in an embodiment of the present application;

fig. 4 is a fourth flowchart illustrating a method for determining a security level in an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an apparatus for determining a security level in an embodiment of the present application;

fig. 6 is a schematic structural diagram of a computer device in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

The embodiment of the application provides a method for determining a security level. The method can be applied to a device for determining the security level. In practical applications, the apparatus for determining a security level may be applied to a computer device, and the computer device may include: a processor, a memory including a stored computer program, the processor being operable to invoke program instructions in the memory to perform the steps of the method of determining a security level.

Fig. 1 is a first flowchart illustrating a method for determining a security level in an embodiment of the present application, and referring to fig. 1, the method for determining a security level may include:

s101: sending the problems in the preset problem library to the artificial intelligence product to be classified;

s102: acquiring reply information of questions in a preset question bank output by an artificial intelligence product;

specifically, in order to evaluate the security degree of the artificial intelligence product to determine the security level of the artificial intelligence product, a dialog process between a user and the artificial intelligence product needs to be simulated, and then, a question in a preset question bank can be input into the artificial intelligence product to be classified, so that reply information aiming at the question output by the artificial intelligence product can be automatically acquired, and the security level of the artificial intelligence product can be determined according to the reply information.

In practical applications, in order to improve the accuracy of the security level of the artificial intelligence product, a large amount of reply information needs to be collected as analysis data, and thus, in the specific implementation process, the preset problem library may be composed of a large amount of standard problems preset by professional graders. Therefore, the accuracy of the safety level of the artificial intelligence product can be improved, and the consistency of the standard in the process of determining the safety level is also ensured.

As an example, in order to improve the accuracy of the security level of the artificial intelligence product, the artificial intelligence product needs to be evaluated more comprehensively, and then all the questions in the preset question bank can be sent to the artificial intelligence product to be classified one by one; in order to increase the speed of determining the security level of the artificial intelligence product, the number of reply messages to be acquired and analyzed needs to be reduced while ensuring certain accuracy, and then a preset number of questions can be randomly extracted from the preset question bank and input into the artificial intelligence product to be classified. The method can be set by a person skilled in the art according to the actual application, and the embodiment of the present application is not specifically set here.

In practical application, the artificial intelligence product to be classified can be a voice intelligence assistant, an intelligent chat robot, an intelligent customer service and the like according to different application scenes of the artificial intelligence product. Of course, other types of artificial intelligence products, such as an intelligent sound box, may also be used, and the embodiments of the present application are not specifically limited herein.

In practical applications, the artificial intelligence products to be classified may include two types, i.e., an online artificial intelligence product and an offline artificial intelligence product, according to the existence of the artificial intelligence products, and then, in a specific implementation process, the above S101 may exist in, but is not limited to, the following two implementations.

In a first implementation manner, if the artificial intelligence product to be classified is an online artificial intelligence product, then the step S101 may include: and sending the questions in the preset question bank to the artificial intelligence product to be graded in a text mode.

In a second implementation manner, if the artificial intelligence product to be classified is an offline artificial intelligence product, then the step S101 may include: and sending the problems in the preset problem library to the artificial intelligence product to be graded in a voice mode.

S103: a security level of the artificial intelligence product is determined based on the reply information.

Specifically, after reply information of the artificial intelligence product to a question in a preset question bank is acquired, the artificial intelligence product can be classified according to the reply information, and the security level of the artificial intelligence product is determined. In this way, the security level of the artificial intelligence product is obtained. Then, in the process of determining the security level of the artificial intelligence product, compared with the manual mode, on one hand, because a large number of problems can be input into the artificial intelligence product, a large amount of rich reply information can be obtained from the artificial intelligence product, then, the security level of the artificial intelligence product is determined based on the large amount of rich reply information, the accuracy of the security level of the artificial intelligence product can be improved, on the other hand, because professional graders are not needed to determine the security level of the artificial intelligence product, then, the problem of standard inconsistency caused by human subjectivity can be reduced, and the problem of standard inconsistency in the process of determining the security level can be effectively avoided.

As an example, the security levels corresponding to the artificial intelligence products can be divided into 4 levels, and the levels can be sorted into 1 level, 2 levels, 3 levels and 4 levels according to the order of the levels from high to low. The 4 levels of security are explained in detail below.

(1) When the safety level of the artificial intelligence product is level 1, the artificial intelligence product is indicated to be of a common level, the safety is highest, and in the process of using the artificial intelligence product, one or more of the following contents may be involved in the reply message output by the artificial intelligence product: the ability to play music for the user, send the user a lovely emoticon, not involve controllable or unimportant user interaction behavior or user content, etc. Of course, the artificial intelligence product with the security level of level 1 may also relate to other contents besides those listed, and the embodiment of the present application is not particularly limited herein.

(2) When the safety level of the artificial intelligence product is 2, the artificial intelligence product is shown to be tutoring level, the safety is high, and in the process of using the artificial intelligence product, one or more of the following contents may be involved in the reply message output by the artificial intelligence product: the ability to identify potential hazards in the user's context and provide help, provide the user with the necessary physical health supervision (e.g., detecting elderly falls) and advice, detect user mental health issues (e.g., depression) and provide help, and the like. Of course, the artificial intelligence product with the security level 2 may also relate to other contents besides the listed contents, and the embodiment of the present application is not particularly limited.

(3) When the safety level of the artificial intelligence product is 3, the artificial intelligence product is adult-grade, the safety is low, and one or more of the following contents may be involved in the reply message output by the artificial intelligence product in the process of using the artificial intelligence product: mild and minor terrorist content, mild or minor sensitive subject matter (e.g., ethnic discrimination, political guidance, etc.), mild and low frequency economic consumption or advertising-induced behavior, mature subject matter (e.g., tobacco use, alcohol use), etc. Of course, the artificial intelligence product with a security level of 3 may also relate to other contents besides those listed, and the embodiment of the present application is not particularly limited herein.

(4) When the safety level of the artificial intelligence product is 4, the artificial intelligence product is indicated to be in a limiting level, the safety is the lowest, and one or more of the following contents may be involved in the reply message output by the artificial intelligence product in the process of using the artificial intelligence product: the system comprises a large amount of bad information, a large amount or high frequency of economic consumption or advertisement inducing behaviors, privacy behaviors such as important individual conversation or individual information disclosure and the like, wherein the bad information can be illegal medicines, low popular erotic contents, dirty words, violence, sensitive subjects (such as ethnic discrimination, political guidance and the like), legal violations, moral violations and the like. Of course, the artificial intelligence product with a security level of 4 may also relate to other contents besides those listed, and the embodiment of the present application is not particularly limited herein.

Of course, in addition to the levels of the security levels exemplarily listed above, other levels may also be used, and here, the embodiment of the present application does not specifically limit the specific levels of the security levels.

How to determine the security level of the artificial intelligence product based on the reply information is described in detail below with specific examples.

First, a method for implementing S103 will be described.

In the implementation process, the above S103 may be implemented by, but not limited to, the following method according to the adopted method for determining the security level.

In a first implementation method, the step S103 may include the following steps a1 to a 2:

step A1: performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result;

step A2: based on the text analysis results, a security level of the artificial intelligence product is determined.

In a second implementation method, the step S103 may include the following steps B1 to B3:

step B1: performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result;

step B2: performing semantic understanding on the reply information to obtain a semantic understanding result;

step B3: and determining the safety level of the artificial intelligence product based on the semantic understanding result and the text analysis result.

In a third implementation method, the step S103 may include the following steps C1 to C2:

step C1: performing semantic understanding on the reply information to obtain a semantic understanding result;

step C2: and determining the safety level of the artificial intelligence product based on the semantic understanding result.

Of course, besides the above-listed implementation methods, the method for determining the security level of the artificial intelligence product based on the reply information may also be other methods, for example, a combination of a preset classification standard and a manual secondary confirmation manner is used, and only the artificial intelligence product with a lower security level is handed over to the grader for secondary determination. The method can be set by a person skilled in the art according to the actual application, and the embodiment of the present application is not limited in detail herein.

Next, a description will be given of a classification criterion set in advance.

In practical applications, in order to ensure the accuracy and authority of the security level of the artificial intelligence product, generally, the preset grading standard is preset by professional graders.

As an example, in other embodiments of the present application, the preset ranking criteria may include: presetting a keyword and a preset mapping relation, wherein the preset mapping relation is the mapping relation between the frequency of the keyword appearing in the reply information and the security level.

In practical application, in order to improve the accuracy of the safety level of the artificial intelligence product, the artificial intelligence product needs to be evaluated relatively comprehensively, so that a plurality of keywords can be set when the preset keywords are set, and thus the number of the preset keywords can be multiple. Thus, the previously set ranking criteria may include: and mapping relations between the plurality of word frequencies and the security levels, which correspond to the plurality of preset keywords one by one respectively. Of course, the preset ranking criteria may also be in other forms, such as a plurality of preset keywords and a comprehensive mapping relationship, where the mapping relationship is a corresponding relationship between the frequency of occurrence of the preset keywords in the reply message and the security level. The method can be set by a person skilled in the art according to practical situations, and the embodiments of the present application are not limited in detail.

Here, the word frequency corresponding to each preset keyword may be a frequency of the preset keyword appearing in the reply message, that is, the word frequency corresponding to each preset keyword may be a ratio between a number of times of the preset keyword appearing in the reply message and a number of messages in the reply message.

For example, the preset keywords include: two keywords, keyword 1 and keyword 2, the security level includes: for example, 4 levels, i.e. level 1, level 2, level 3 and level 4, the preset ranking criteria may include: keyword 1 and keyword 2, mapping relation 1 shown in table 1-1 below corresponding to keyword 1, and mapping relation 2 shown in table 1-2 below corresponding to keyword 2. Alternatively, the preset classification criterion may include: keyword 1 and keyword 2, and the mapping relationships shown in tables 1 to 3 below, which correspond to both keyword 1 and keyword 2.

Word frequency	Level of security
		The frequency of the keyword 1 appearing in the reply message is less than or equal to the word frequency 1	Level 1
Word frequency 1<The frequency of the keyword 1 appearing in the reply message is less than or equal to the word frequency 2	Stage 2
		Word frequency 2<The frequency of the keyword 1 appearing in the reply message is less than or equal to the word frequency 3	Grade 3
Frequency of occurrence of keyword 1 in reply message>Word frequency 3	4 stage

TABLE 1-1 mapping relationship corresponding to keyword 1

Word frequency	Level of security
		The frequency of the keyword 2 appearing in the reply message is less than or equal to the word frequency 4	Level 1
Word frequency 4<The frequency of the keyword 2 appearing in the reply message is less than or equal to the word frequency 5	Stage 2
		Word frequency 5<The frequency of the keyword 2 appearing in the reply message is less than or equal to the word frequency 6	Grade 3
Frequency of occurrence of keyword 2 in reply message>Word frequency 6	4 stage

TABLE 1-2 mapping relationships corresponding to keyword 2

Tables 1-3 Integrated mapping relationships

Of course, besides the mapping relationships listed above, the mapping relationship between the word frequency and the security level may also be in other forms, for example, as long as a certain keyword appears, no matter how much the word frequency appears, the mapping relationship corresponds to level 4, and here, the embodiment of the present application is not particularly limited.

Then, how to determine the security level of the artificial intelligence product based on the preset ranking criteria will be described in detail.

As an example, the following are included with the preset ranking criteria: the mapping relationship between the predetermined keywords and their corresponding word frequencies and security levels is an example, and how to determine the security level of the artificial intelligence product based on the predetermined grading standard will be described in detail.

In other embodiments of the present application, the step a1 or the step B1 may include: acquiring each preset keyword from a preset grading standard; and determining the frequency of each preset keyword appearing in the reply message as a text analysis result.

Specifically, when the preset hierarchical standard includes a plurality of preset keywords, and text analysis is performed on the reply information according to the preset hierarchical standard, each preset keyword may be obtained from the preset hierarchical standard, the frequency of occurrence of each preset keyword in the reply information is determined, and then the frequency of occurrence of each preset keyword in the reply information is determined as the text analysis result corresponding to the reply information.

In other embodiments of the present application, the step a2 may include: and determining the safety level of the artificial intelligence product based on the frequency of each preset keyword appearing in the reply information and the mapping relation between the word frequency and the safety level corresponding to each preset keyword.

As an example, in order to avoid bringing security risks to users, the security level of the artificial intelligence product may be determined in a strict manner, and specifically, when the preset hierarchical standard includes a plurality of preset keywords, a plurality of word frequencies may be obtained, so that a plurality of security levels may be determined according to a mapping relationship between the word frequencies and the security levels, and then, in order to finally determine the security level corresponding to the artificial intelligence product, the security level corresponding to the frequency of each preset keyword appearing in the reply information may be determined according to the mapping relationship between the word frequency and the security level corresponding to each preset keyword, so as to obtain a plurality of candidate levels corresponding to the artificial intelligence product; and determining the security level of the artificial intelligence product based on the candidate level with the lowest security level in the plurality of candidate levels.

For example, assuming that the preset ranking criteria includes 8 keywords, 8 candidate ranks may be determined according to the word frequencies corresponding to the 8 keywords one by one, including: "level 1, level 3, level 4, level 2, level 3, level 2", it can be seen that the candidate level with the lowest security level among the 8 candidate levels is level 4, and then level 4 may be determined as the security level of the artificial intelligence product.

As an example, when the preset ranking criteria includes a plurality of preset keywords, a plurality of word frequencies may be obtained, and thus, according to a mapping relationship between the word frequencies and the security levels, a plurality of security levels may be determined, and then, in order to finally determine the security level corresponding to the artificial intelligence product, according to a mapping relationship between the word frequencies and the security levels corresponding to each of the preset keywords, a security level corresponding to a frequency of occurrence of each of the preset keywords in the reply information may be determined, a plurality of candidate levels corresponding to the artificial intelligence product may be obtained, and then, a candidate level with a largest number of occurrences among the plurality of candidate levels may be determined as the security level corresponding to the artificial intelligence product.

For example, assuming that the preset ranking criteria includes 8 keywords, 8 candidate ranks may be determined according to the word frequencies corresponding to the 8 keywords one by one, including: "level 1, level 3, level 4, level 2, level 1, level 2", it can be seen that, the candidate level of the 8 candidate levels with the occurrence number is level 2, and then level 2 can be determined as the security level of the artificial intelligence product. In addition, since level 4 (the level with the lowest security) appears in the candidate levels, in order to avoid bringing security risks to the user, a security prompt can be performed based on the security level, which indicates that the artificial intelligence product may have a certain security risk.

Of course, in addition to the two examples listed above, other solutions may be used to determine the security level of the artificial intelligence product according to a plurality of candidate levels. The method can be set by a person skilled in the art according to a specific practical application, and the embodiments of the present application are not limited in detail herein.

In another embodiment of the present application, in order to improve the speed of text analysis of the reply message, the step a1 or the step B1 may include: matching a plurality of preset keywords with the reply information; determining a target keyword appearing in the reply information in the plurality of preset keywords according to the matching result, and counting the frequency of the target keyword appearing in the reply information; and determining the frequency of the statistical target keywords appearing in the reply message as a text analysis result.

Specifically, when the preset ranking criteria include a plurality of preset keywords, when text analysis needs to be performed on the reply information according to the preset ranking criteria, the preset keywords may be matched with the reply information, and then after a matching result is obtained, which target keywords appear in the reply information among the preset keywords may be counted according to the matching result, and the frequency of the target keywords appearing in the reply information may be counted. In this way, the desired text analysis results are obtained.

As an example, in order to improve the matching speed, the step of matching the plurality of preset keywords with the reply information may include: performing word segmentation processing on the reply information, and dividing the reply information into a plurality of terms; and matching each term with a plurality of preset keywords respectively to obtain a matching result.

In an embodiment of the present application, the step a2 may include: and determining the safety level of the artificial intelligence product based on the frequency of the target keyword appearing in the reply information and the mapping relation between the corresponding word frequency and the safety level.

For example, if it is determined that the number of target keywords appearing in the reply message is multiple in the multiple preset keywords after the matching is performed, the step a2 may include: determining the security level corresponding to the frequency of each target keyword appearing in the reply information according to the mapping relation between the word frequency and the security level corresponding to each target keyword, and obtaining a plurality of candidate levels corresponding to the artificial intelligence product; the security level of the artificial intelligence product is determined based on at least one of the candidate levels having the lowest security level or the candidate level having the highest number of occurrences of the candidate levels.

Of course, besides the two implementation manners listed above, when the preset ranking criteria include a plurality of preset keywords, the security level of the artificial intelligence product may also be determined in other manners, and the embodiment of the present application is not specifically limited herein.

Finally, how to determine the safety level of the artificial intelligence product based on the semantic understanding result and the text analysis result is explained.

In other embodiments of the present application, the step B3 may include: determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result; determining a second security level corresponding to the artificial intelligence product based on the text analysis result; a security level of the artificial intelligence product is determined based on the first security level and the second security level.

By way of example, still at a security level: for example, 4 levels, i.e., level 1, level 2, level 3, and level 4, in order to determine the first security level corresponding to the artificial intelligence product according to the semantic understanding result, a mapping relationship between the semantic understanding result and the security level shown in table 2 below may be preset, and after the semantic understanding result corresponding to the reply information is obtained, the corresponding security level may be determined as the first security level corresponding to the artificial intelligence product based on the mapping relationship.

Table 2 mapping relationship between semantic understanding result and security level

In addition, in practical application, the obtained semantic understanding result may include a plurality of semantic understanding results according to different amounts or contents of information in the reply information, and then a plurality of candidate levels may be determined according to a mapping relationship between the semantic understanding result and the security level. For example, the candidate level with the lowest security level among the plurality of candidate levels may be determined as the first security level, or the candidate level with the highest occurrence number among the plurality of candidate levels may be determined as the first security level.

Of course, in addition to the above listed implementation manners, other manners may be adopted to determine the first security level corresponding to the artificial intelligence product based on the semantic understanding result. Here, the embodiment of the present application is not particularly limited.

In another embodiment of the application, in order to determine a final security level corresponding to an artificial intelligence product according to a first security level corresponding to the artificial intelligence product determined by the semantic understanding result and a second security level corresponding to the artificial intelligence product determined by the text analysis result, it may be determined whether the first security level and the second security level are the same; if the first security level is determined to be different from the second security level, determining the higher one of the first security level and the second security level as the security level of the artificial intelligence product; if it is determined that the first security level and the second security level are the same, any one of the first security level and the second security level may be determined as the security level of the artificial intelligence product.

In another embodiment of the present application, in order to facilitate the user to know the security level of each artificial intelligence product to select a suitable artificial intelligence product, the obtained security level of the artificial intelligence product needs to be displayed to the user, and then, in the implementation process, after S103, the method may further include: and displaying the safety level of the artificial intelligence product.

Specifically, in order to facilitate the user to know the security level of the artificial intelligence product, the security level of the artificial intelligence product can be displayed to the user after the artificial intelligence product is classified.

In another embodiment of the present application, in order to facilitate user-defined setting of the security level of the artificial intelligence product, after S103, the method may further include: displaying the security level of the artificial intelligence product; responding to the modification operation of the user on the security level, and acquiring the security level input by the user aiming at the artificial intelligence product; the security level of the artificial intelligence product is modified to the security level entered by the user.

Particularly, in order to improve the accuracy of determining the safety level of the artificial intelligence product and improve the user experience, after the artificial intelligence product is classified, the safety level of the artificial intelligence product can be displayed to a user, the user can flexibly select the safety level obtained by receiving automatic classification according to the actual use condition of the user, the user can also flexibly select the safety level to be modified, and the safety level corresponding to the artificial intelligence product obtained by automatic classification is modified into the safety level set by the user.

Thus, the process of grading the artificial intelligence products based on the reply information is completed.

As can be seen from the above, in the method for determining a security level provided in the embodiment of the present application, after sending a question in a preset question bank to an artificial intelligence product to be classified, reply information for the question output by the artificial intelligence product can be automatically obtained, and then, the artificial intelligence product is classified based on the reply information to obtain the security level of the artificial intelligence product. Therefore, because the reply information of the artificial intelligence product to the problems in the question bank is automatically acquired through the preset question bank, and the safety level of the artificial intelligence product is determined according to the reply information, the accuracy of the safety level of the artificial intelligence product can be improved, and the problem of inconsistent standards in the process of determining the safety level can be effectively avoided.

Based on the foregoing embodiments, the present application provides a method for determining a security level. The method can be applied to the following scenes: after the artificial intelligence product is interacted with the problems in the preset problem library, the artificial intelligence product is graded at least through the text analysis result corresponding to the reply information returned by the artificial intelligence product, and the safety level of the artificial intelligence product is determined.

Fig. 2 is a schematic flowchart of a second method for determining a security level in an embodiment of the present application, and referring to fig. 2, the method for determining a security level may include:

s201: sending the problems in the preset problem library to the artificial intelligence product to be classified;

s202: acquiring reply information of a problem output by an artificial intelligence product;

s203: performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result;

s204: a security level of the artificial intelligence product is determined based at least on the text analysis results.

As an example, in a specific implementation process, the step S204 may include: performing semantic understanding on the reply information to obtain a semantic understanding result; determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result; determining a second security level corresponding to the artificial intelligence product based on the text analysis result; a security level of the artificial intelligence product is determined based on the first security level and the second security level.

As an example, in a specific implementation process, the step S204 may include: determining a second security level corresponding to the artificial intelligence product based on the text analysis result; the second security level is determined as the security level of the artificial intelligence product.

As can be seen from the above, in the method for determining a security level provided in this embodiment of the present application, after sending a question in a preset question bank to an artificial intelligence product to be classified, reply information for the question output by the artificial intelligence product can be automatically obtained, then, text analysis is performed on the reply information based on a preset classification standard to obtain a text analysis result, and finally, the artificial intelligence product can be classified at least based on the text analysis result to obtain the security level of the artificial intelligence product. In this way, because the reply information of the artificial intelligence product to the problems in the question bank is automatically acquired through the preset question bank, and the safety level of the artificial intelligence product is automatically determined according to the text analysis result corresponding to the reply information, the accuracy of the safety level of the artificial intelligence product can be improved, and the problem of inconsistent standards in the process of determining the safety level can be effectively avoided.

Based on the foregoing embodiments, the present application provides a method for determining a security level. The method can be applied to the following scenes: after a dialogue is carried out between the artificial intelligence product and the questions in the preset question bank, the artificial intelligence product is graded at least through a semantic understanding result corresponding to reply information returned by the artificial intelligence product, so that the safety level of the artificial intelligence product is determined.

Fig. 3 is a third schematic flowchart of a method for determining a security level in an embodiment of the present application, and referring to fig. 3, the method for determining a security level may include:

s301: sending the problems in the preset problem library to the artificial intelligence product to be classified;

s302: acquiring reply information of a problem output by an artificial intelligence product;

s303: performing semantic understanding on the reply information to obtain a semantic understanding result;

s304: a security level of the artificial intelligence product is determined based at least on the semantic understanding result.

As an example, in a specific implementation process, the step S304 may include: determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result; performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result; determining a second security level corresponding to the artificial intelligence product based on the text analysis result; a security level of the artificial intelligence product is determined based on the first security level and the second security level.

As an example, in a specific implementation process, the step S304 may include: determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result; the first security level is determined as a security level of the artificial intelligence product.

As can be seen from the above, in the method for determining a security level provided in this embodiment of the application, after sending a problem in a preset problem library to an artificial intelligence product to be classified, reply information for the problem output by the artificial intelligence product can be automatically obtained, then, semantic understanding is performed on the reply information to obtain a semantic understanding result corresponding to the reply information, and finally, the artificial intelligence product can be classified at least based on the semantic understanding result to obtain the security level of the artificial intelligence product. In this way, because the reply information of the artificial intelligence product to the problems in the question bank is automatically acquired through the preset question bank, and the safety level of the artificial intelligence product is automatically determined according to the semantic understanding result corresponding to the reply information, the accuracy of the safety level of the artificial intelligence product can be improved, and the problem of inconsistent standards in the process of determining the safety level can be effectively avoided.

Based on the foregoing embodiments, the present application provides a method for determining a security level. The method can be applied to the following scenes: after the artificial intelligence product is interacted with the problems in the preset problem base, the artificial intelligence product is graded through a text analysis result and a semantic understanding result corresponding to the reply information returned by the artificial intelligence product, so that the safety level of the artificial intelligence product is determined.

Fig. 4 is a fourth flowchart illustrating a method for determining a security level in an embodiment of the present application, and referring to fig. 4, the method for determining a security level may include:

s401: sending the problems in the preset problem library to the artificial intelligence product to be classified;

s402: acquiring reply information of a problem output by an artificial intelligence product;

s403: performing semantic understanding on the reply information to obtain a semantic understanding result;

s404: determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result;

s405: performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result;

s406: determining a second security level corresponding to the artificial intelligence product based on the text analysis result;

s407: a security level of the artificial intelligence product is determined based on the first security level and the second security level.

As can be seen from the above, the method for determining a security level provided in the embodiment of the present application can automatically obtain reply information for a question output by an artificial intelligence product after sending the question in a preset question bank to the artificial intelligence product to be classified. Then, semantic understanding can be performed on the reply information to obtain a semantic understanding result corresponding to the reply information, a first security level corresponding to the artificial intelligence product is determined based on the semantic understanding result, and text analysis can be performed on the reply information based on a preset grading standard to obtain a text analysis result corresponding to the reply information. Finally, a security level corresponding to the artificial intelligence product can be determined based on the first security level and the second security level. In this way, because the reply information of the artificial intelligence product to the problems in the question bank is automatically acquired through the preset question bank, and the safety level of the artificial intelligence product is automatically determined according to the semantic understanding result corresponding to the reply information, the accuracy of the safety level of the artificial intelligence product can be improved, and the problem of inconsistent standards in the process of determining the safety level can be effectively avoided. In addition, the safety level of the artificial intelligence product is determined based on the text analysis result and the semantic understanding result corresponding to the reply information, so that the accuracy of the safety level of the artificial intelligence product can be greatly improved.

Based on the same inventive concept, as an implementation of the method, the embodiment of the application provides a device for determining a security level. Fig. 5 is a schematic structural diagram of an apparatus for determining a security level in an embodiment of the present application, and referring to fig. 5, the apparatus 50 may include: a sending module 501, configured to send a preset question in a question bank to an artificial intelligence product to be classified; an obtaining module 502, configured to obtain reply information of a question output by an artificial intelligence product; a determining module 503 for determining a security level of the artificial intelligence product based on the reply information.

In the embodiment of the application, the determining module is used for performing text analysis on the reply information based on a preset grading standard to obtain a text analysis result; a security level of the artificial intelligence product is determined based at least on the text analysis results.

In the embodiment of the application, the determining module is used for performing semantic understanding on the reply information to obtain a semantic understanding result; and determining the safety level of the artificial intelligence product based on the semantic understanding result and the text analysis result.

In the embodiment of the application, the determining module is used for determining a first security level corresponding to the artificial intelligence product based on the semantic understanding result; determining a second security level corresponding to the artificial intelligence product based on the text analysis result; a security level of the artificial intelligence product is determined based on the first security level and the second security level.

In an embodiment of the present application, the preset classification criteria may include: and mapping relations between the preset keywords and the corresponding word frequencies and the security levels of the preset keywords.

In the embodiment of the application, the determining module is used for matching a plurality of preset keywords with the reply information; determining a target keyword appearing in the reply information in the plurality of preset keywords according to the matching result, and counting the frequency of the target keyword appearing in the reply information; determining the frequency of the statistical target keywords appearing in the reply information as a text analysis result; and the method is also used for determining the security level of the artificial intelligence product at least based on the frequency of the target keywords appearing in the reply message and the mapping relation between the corresponding word frequency and the security level.

In the embodiment of the application, the determining module is configured to determine, according to a mapping relationship between a word frequency and a security level corresponding to each target keyword, the security level corresponding to the frequency of occurrence of each target keyword in the reply message, if the number of the target keywords is multiple, and obtain multiple candidate levels corresponding to the artificial intelligence product; the security level of the artificial intelligence product is determined based on at least one of the candidate levels having the lowest security level or the candidate level having the highest number of occurrences of the candidate levels.

In the embodiment of the application, the determining module is used for performing semantic understanding on the reply information to obtain a semantic understanding result; a security level of the artificial intelligence product is determined based at least on the semantic understanding result.

In other embodiments of the present application, the apparatus may further include: the safety level display module is used for displaying the safety level of the artificial intelligence product; the safety level acquisition module is used for responding to the modification operation of the user on the safety level and acquiring the safety level input by the user aiming at the artificial intelligence product; and the safety level modification module is used for modifying the safety level of the artificial intelligence product into the safety level input by the user.

Based on the same inventive concept, the embodiment of the application provides computer equipment. Fig. 6 is a schematic structural diagram of a computer device in an embodiment of the present application, and referring to fig. 6, the computer device 60 includes: at least one processor 61; and at least one memory 62, bus 63 connected to the processor 61; wherein, the processor 61 and the memory 62 complete mutual communication through a bus 63; the processor 61 is arranged to call program instructions in the memory 62 to perform the steps of the method of determining a security level in one or more of the embodiments described above.

Accordingly, based on the same inventive concept, embodiments of the present application further provide a processor, where the processor is configured to execute a program, where the program executes the steps of the method for determining a security level in one or more embodiments described above.

The Processor may be implemented by a Central Processing Unit (CPU), a MicroProcessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like. The Memory may include volatile Memory in a computer readable medium, Random Access Memory (RAM), and/or nonvolatile Memory such as Read Only Memory (ROM) or Flash Memory (Flash RAM), and the Memory includes at least one Memory chip.

It should be noted that, in the embodiments of the present application, if the method for determining the security level in one or more embodiments is implemented in the form of a software functional module and sold or used as a stand-alone product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present application.

Accordingly, based on the same inventive concept, embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and where the program, when executed, controls a computer device in which the storage medium is located to perform the steps of the method for determining a security level in one or more embodiments described above.

Here, it should be noted that: the above description of the apparatus, computer device or computer-readable storage medium embodiments is similar to the description of the method embodiments above, with similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus, the computer device or the computer-readable storage medium of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

The present application describes embodiments, but the description is illustrative rather than limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or instead of any other feature or element in any other embodiment, unless expressly limited otherwise.

The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive concept as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive aspects to form yet another unique inventive aspect, as defined by the claims. Thus, it should be understood that any of the features shown and/or discussed in this application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not limited except as by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

Further, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other orders of steps are possible as will be understood by those of ordinary skill in the art. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. Further, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.

It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.