Method for automated action of an electronic device
阅读说明:本技术 用于电子设备的自动化动作的方法 (Method for automated action of an electronic device ) 是由 维贾伊·斯里尼瓦桑 克里斯蒂安·克耶勒 金红霞 于 2018-08-24 设计创作,主要内容包括:一种用于动作自动化的方法,包括使用电子设备基于域信息来确定动作。获取与动作相关联的活动模式。对于每个活动模式,确定候选动作规则。每个候选动作规则指定动作发生时的一个或更多个前提条件。从多个候选动作规则中确定用于动作自动化的一个或更多个优选的候选动作规则。(A method for action automation includes determining, using an electronic device, an action based on domain information. An activity pattern associated with the action is obtained. For each active mode, a candidate action rule is determined. Each candidate action rule specifies one or more preconditions for when an action occurs. One or more preferred candidate action rules for action automation are determined from the plurality of candidate action rules.)
1. A method for action automation, the method comprising:
determining, using the electronic device, an action based on the domain information;
obtaining a plurality of activity patterns associated with the action;
for each activity pattern, determining candidate action rules, wherein each candidate action rule specifies one or more preconditions at which the action occurs; and
one or more preferred candidate action rules for the action automation are determined from a plurality of candidate action rules.
2. The method for action automation of claim 1, wherein the one or more preferred candidate action rules include one or more preconditions that cause the action to occur with a probability of exceeding a first likelihood threshold.
3. The method for action automation of claim 1, wherein the one or more preconditions are associated with a first duration of time whose frequency of occurrence in a second duration of time is greater than a second threshold and less than a third threshold.
4. The method for action automation according to claim 3, wherein said one or more preferred candidate action rules cover a number of occurrences of said action within said second duration, which is greater than a fourth threshold.
5. The method for action automation of claim 1, wherein a total number of the one or more preferred candidate action rules is less than a fifth threshold.
6. The method for action automation of claim 1, wherein the one or more preferred candidate action rules include a subset of candidate action rules that summarizes all candidate action rules.
7. The method for action automation of claim 1, the method further comprising:
providing a suggestion on the electronic device for automating the action based on the one or more preferred candidate action rules.
8. The method for action automation of claim 1, the method further comprising:
providing an automated reminder to perform the action on the electronic device based on the one or more preferred candidate action rules.
9. An electronic device, the electronic device comprising:
a memory storing instructions; and
at least one processor that executes the instructions comprising a process configured to:
determining an action based on the domain information;
obtaining a plurality of activity patterns associated with the action;
for each activity pattern, extracting candidate action rules, wherein each candidate action rule specifies one or more preconditions when the action occurs;
one or more preferred candidate action rules for the action automation are determined from a plurality of candidate action rules.
10. The electronic device of claim 9, wherein the one or more preferred candidate action rules include preconditions that cause the action to occur with a probability of exceeding a first likelihood threshold.
11. The electronic device of claim 9, wherein the one or more preconditions are associated with a first duration having a frequency of occurrence in a second duration that is greater than a second threshold and less than a third threshold.
12. The electronic device of claim 11, wherein the one or more preferred candidate action rules encompass a number of occurrences of the action within the second duration that is greater than a fourth threshold.
13. The electronic device of claim 9, wherein:
the total number of the one or more preferred candidate action rules is less than a fifth threshold; and
the one or more preferred candidate action rules include a subset of candidate action rules summarizing all candidate action rules.
14. The electronic device of claim 9, wherein the processor is further configured to:
providing a suggestion on the electronic device for automating the action based on the one or more preferred candidate action rules; and
providing an automated reminder to perform the action on the electronic device based on the one or more preferred candidate action rules.
15. A non-transitory processor-readable medium comprising a program which, when executed by a processor, performs a method comprising:
determining, using the electronic device, an action based on the domain information;
obtaining a plurality of activity patterns associated with the action;
for each activity pattern, extracting candidate action rules, wherein each candidate action rule specifies one or more preconditions when the action occurs; and
one or more preferred candidate action rules for the action automation are determined from a plurality of candidate action rules.
Technical Field
One or more embodiments relate generally to smart device automation, and more particularly to determining personalized action automation.
Background
Modern smartphones and ubiquitous computing systems collect a large amount of contextual data from users. Conditional action rules, popular with IFTTT (If-This-Then-eat) platforms, are a popular way to let users automatically perform frequently repetitive tasks or receive intelligent reminders, thanks to the understandability and control That the rules provide to the user. The main drawback of IFTTT systems is that they burden the user to manually specify action rules.
Disclosure of Invention
Solution to the problem
One or more embodiments are generally directed to determining preferred rules for action automation. In one embodiment, a method includes determining, using an electronic device, an action based on domain information. An activity pattern associated with the action is obtained. For each active mode, a candidate action rule is determined. Each candidate action rule specifies one or more preconditions for when an action occurs. One or more preferred candidate action rules for the action automation are determined from a plurality of candidate action rules.
In another embodiment, an electronic device includes a memory storing instructions. The at least one processor executes instructions comprising a process configured to: the method includes determining an action based on domain information, obtaining a plurality of activity patterns associated with the action, extracting candidate action rules for each activity pattern, wherein each candidate action rule specifies one or more preconditions when the action occurs, and determining one or more preferred candidate action rules for automation of the action from the plurality of candidate action rules.
In one embodiment, a non-transitory processor-readable medium includes a program that, when executed by a processor, performs a method that includes determining, using an electronic device, an action based on domain information. A plurality of activity patterns associated with the action are obtained. For each active pattern, candidate action rules are extracted. Each candidate action rule specifies one or more preconditions for when the action occurs. One or more preferred candidate action rules for the action automation are determined from a plurality of candidate action rules.
The foregoing and other aspects and advantages of one or more embodiments will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of one or more embodiments.
Drawings
For a fuller understanding of the nature and advantages of the embodiments, as well as the preferred mode of use, reference should be made to the following detailed description read in conjunction with the accompanying drawings, in which:
fig. 1 illustrates a schematic diagram of a communication system, in accordance with some embodiments;
FIG. 2 illustrates a block diagram of an architecture of a system including an electronic device including a rule selector processing application, in accordance with some embodiments;
FIG. 3 illustrates a rule selector process flow according to some embodiments;
FIG. 4 illustrates an example table of rule selection metrics used in the rule selector process in accordance with some embodiments;
FIG. 5 illustrates a reduction and recommendation of rule selection according to some embodiments;
FIG. 6 illustrates a table of example universal rules across populations, in accordance with some embodiments;
FIG. 7 illustrates an example interface for adding rules, according to some embodiments;
FIG. 8 illustrates an example interface for saved rules, according to some embodiments;
FIG. 9 illustrates an example interface for modifying rules, according to some embodiments;
FIG. 10 illustrates an example table of intelligent Television (TV) rules for an example user, in accordance with some embodiments;
FIG. 11 illustrates an example table of context driven rules for an example user, in accordance with some embodiments;
12A-12C illustrate smart device usage for rule setting, rule invocation, and personalized reminders, according to some embodiments;
13A-13B illustrate smart device usage for automating user actions by deriving complex personalization rules, according to some embodiments;
14A-14C illustrate smart device usage for predicting presentation of user actions and contextual action cards/shortcuts, in accordance with some embodiments;
FIG. 15 illustrates a block diagram of a process for determining preferred rules for action automation, in accordance with some embodiments;
FIG. 16 is a high-level block diagram illustrating an information handling system including a computing system implementing one or more embodiments.
Detailed Description
The following description is made for the purpose of illustrating the general principles of one or more embodiments and is not meant to limit the inventive concepts claimed herein. In addition, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations. Unless otherwise expressly defined herein, all terms are to be given the broadest interpretation so as to encompass both implicit meanings from the description as well as meanings understood by those skilled in the art and/or as defined in dictionaries, monographs, etc.
It should be noted that the term "at least one" refers to one or more of the following elements. For example, "at least one of a, b, c, or a combination thereof" may be interpreted as "a", "b", or "c", respectively, alone; or "a" and "b" are combined together, or "b" and "c" are combined together; "a" and "c" are combined together; or "a", "b", and "c" are combined together.
One or more embodiments provide a preference rule that determines automation of an action. Some embodiments include a method comprising determining, using an electronic device, an action based on domain information. An activity pattern associated with the action is obtained. For each active mode, a candidate action rule is determined. Each candidate action rule specifies one or more preconditions for when an action occurs. One or more preferred candidate action rules are determined from a plurality of candidate action rules for action automation.
In some embodiments, the problem of how to present a small subset of rules to a smartphone user and allow the smartphone user to interactively select action rules is addressed by designing and implementing a rule selector process that provides the smartphone (or functionally similar smart device) user with an interactive rule selection tool for browsing, modifying, and selecting action rules from a small set of aggregated rules presented to the user.
In some embodiments, a rule selector processes the auto-suggest rules and provides the user with the ability to select and modify suggested rules. Modern ubiquitous computing systems (e.g., smartphones, wearable devices, and smart TVs) collect a large amount of contextual information from users. For example, smartphones collect user scenarios such as the user's daily activities (dining, commuting, exercise, etc.), device activities (such as applications used, application actions performed, Short Message Service (SMS) and call logs), and various granularity of semantic places of the user (such as home, work, cafe, cubicle or living room). Context data may enrich the user experience in a number of ways, such as presenting the most relevant applications to the user that give the current context, recommending events or content based on the user context, and even improving the functionality of the voice assistant by taking advantage of the current context. Context recognition may also enrich the user experience by allowing the user to specify context-aware conditional action rules. Typically, the conditional action rule is of the form: "if certain contextual preconditions are met, please perform some device action". Commercially, conditional action rules have been implemented by IFTTT (If-This-Then-eat), which is a free platform That allows users to specify and execute conditional action rules across popular cell phone platforms and in various areas such as smart home automation, smart phone action automation and reminders, and even task automation of automobiles. The IFTTT rule allows users to automatically perform repetitive tasks or receive intelligent reminders and notifications, explicitly and frequently in appropriate scenarios.
The essential way to automatically perform a device action or display a reminder is to express the question as a user behavior prediction question. For example, if the user is predicted to set the volume to high confidence mute, the smartphone is muted or the user is prompted to do so. The main reason why IFTTT rules are popular among a subset of users is that they provide more control and transparency to end users in the event that action automation or reminders occur, so that they can add and delete rules as needed. However, IFTTT relies entirely on the end user to specify the rules that they wish to automatically execute. This places a burden on the user to carefully specify their own rules based on the memory of their personal behavior patterns. Alternatively, it is not always useful to have a typical action rule base for the average person, since most action rules found for each individual are highly personalized. It should be noted that even for common actions shared among multiple rules, the proportion of unique rules of these common actions shared among sample users is low. In one experiment, 714 unique rules were shown for actions that are commonly shared between users, but only 59 of these unique rules are also shared by all users. Therefore, the vast majority of the rules recommended should be personalized rules that are applicable to a particular user, which strongly motivates the need for personalized and interactive rule selection interfaces. In some embodiments, the rule selector process solves the conventional problem by automatically mining and presenting a small set of aggregated action rules to the user, and enabling the user to interactively select action rules through a rule selection interface.
In some embodiments, the rule selector process provides a complete end-to-end interface for conditional action rule selection that: (i) collect contextual data associated with the user, (ii) mine and select a small number of candidate conditional action rules to maximize the importance of the rules and mitigate the redundancy of the rules, and (iii) provide a process to allow end users to browse, modify, and select mined conditional action rules that they want to automatically execute in daily life. For example, a rule selector process may provide a smart reminder for charging a smartphone while the user is in a stall, while other rule selector processes may provide another rule that automatically executes a music playlist on the smartphone based on the user's behavior pattern. Some embodiments provide for the selection of conditional action rules for co-occurrence (co-occurence).
In some embodiments, the rule selector process provides the smartphone user with the ability to create personalized conditional action rules using candidate rules discovered by pattern mining of mobile context data. To reduce the number of candidate rules generated from user behavior patterns, a rule selector process is implemented to select the most important conditional action rules based on four rule selection metrics, which are easily interpreted by the smartphone user: one rule redundancy measure (total action range), and three rule importance measures (confidence, interval count, and scenario specificity).
In some embodiments, the rule selector process provides interactive rule exploration and selection that provides a smartphone user with a view to a different set of mined conditional action rules based on adjusting the rule selection metrics, focusing on the selection of rules, and modifying the rules if necessary before deciding to incorporate the rules into daily life. Rule selector processing can be applied to public context data sets from smart homes, TV ratings, and digital health monitoring to automate in these areas.
In some embodiments, each action rule recommended has the following form: (preconditions- > actions), where the preconditions can be any combination of the following pattern types: co-occurrence, when A, B and C appear together, perform action D (e.g., remind me to run on saturday morning, sunny day, and when the temperature >20 ℃); a sequence that, after performing A, B and C in sequence, performs action D (e.g., after turning off the bedroom light, set the phone to vibrate, connect the smartphone to the charger, and then play the "sleep" playlist on the phone); periodically and temporally, i last performed actions A, B and C N minutes later, please perform action B (e.g., two hours after arriving at the office, remind me to order food from the delivery service).
In some embodiments, thousands of candidate conditional action rules may be aggregated into a small set of 4-5 conditional action rules by action based on optimizing four rule metrics simultaneously.
1): average rule confidence: for each action rule, the rule confidence is defined as the likelihood that the user will perform the target action when the preconditions are satisfied. That is, the rule confidence is the conditional probability P (action | precondition). Rules with high rule confidence are preferably recommended, i.e., users are more likely to perform automated target actions. For example, if the user plays the Bollywood playlist only 50% of the time commuting home, then the rule { commute home } - > playing music applies the Bollywood playlist, which is a poor choice for rule automation. However, if the user plays his favorite news 95% of the time commuting home, then the rule { commute home } - > playing my favorite news headline is a better choice for rule automation.
2) Average rule specificity: for each action rule, rule specificity is defined as a particular degree of precondition in the action rule. It is desirable to choose a rule that is highly specific and as specific as possible to improve the effectiveness of the timing of the automatic action. That is, the rule specificity is the inverse of the prior probability (1-P (precondition)). For example, the preconditions of { "at home" } are not very explicit and have low specificity because it occurs very frequently and with high a priori probability (lower a priori inverse). On the other hand, the precondition { "in kitchen", "cook breakfast" } has a high specificity because it occurs only once a day in the morning at most. That is, the inverse of the prior probability of { "in kitchen", "cook breakfast" } is high.
3) The total action range is as follows: for our recommended conditional action rule set, the total action range is the percentage of different target user actions covered by the rule set. For example, if two rules are recommended to remind the battery to charge: { in the office, the battery runs low 50% } and { in the office, between 10 am and 11 am } are poor choices, since it covers only 50% of the user's charging actions. However, if the following two battery charge alert action rules are recommended: { at the office, battery level < 50% } and { at home, battery level < 25% }, which is a better choice since it covers 95% of the user charging actions.
4) The number of the rules is as follows: it is desirable to minimize the number of rules recommended to the user while maximizing the three metrics described above. It is desirable to recommend as few rules as possible to the end user to maximize user convenience in terms of action automation and minimize user disruption in the recommendation and enablement of rules.
In some embodiments, maximizing the total action scope while minimizing the number of rules generated is important to ensure that the utility of action automation is maximized to the end user through conditional action rules, while minimizing user annoyance due to excessive rule suggestions. The rule selector processes a set of conditional action rules recommended to the end user to automatically perform any target action while optimizing the following four metrics: maximizing the action range; minimizing the number of rules to minimize user annoyance due to excessive rule suggestions; maximizing the rule confidence; and maximizing the specificity of the rules. Unlike conventional rule aggregation, all conventional work focuses on maximizing the single rule importance metric and/or the total number of modes (mode ranges) covered by the aggregated rule. None of the existing work focuses on maximizing the total number of user target actions (total action range) covered by the generated aggregation rules. One advantage of an embodiment is that the total action covered by the aggregated rules is maximized compared to traditional work focusing on maximizing the pattern scope, which ensures that the generated conditional action rules significantly improve the convenience of the end user in automated actions.
Fig. 1 is a schematic diagram of a
Any suitable circuit, device, system, or combination thereof (e.g., a wireless communication infrastructure including communication towers and telecommunication servers) that may be used to create a communication network may be used to create the
The transmitting
Fig. 2 illustrates a functional block diagram of an
In one embodiment, all applications employed by audio output 123, display 121, input mechanism 124, communication circuitry 125, and microphone 122 may be interconnected and managed by control circuitry 126. In one example, a handheld music player capable of sending music to other tuning devices may be incorporated into the electronic device 120.
In one embodiment, audio output 123 may include any suitable audio component for providing audio to a user of electronic device 120. For example, audio output 123 may include one or more speakers (e.g., mono or stereo speakers) built into electronic device 120. In some embodiments, audio output 123 may include an audio component remotely coupled to electronic device 120. For example, the audio output 123 may include a headset, headphones, or earpieces, which may be coupled to the communication device by a wire (e.g., to the electronic device 120 by a jack) or wirelessly (e.g.,
earphone orA headset) is coupled to the communication device.In one embodiment, display 121 may include any suitable screen or projection system for providing a display visible to a user. For example, the display 121 may include a screen (e.g., an LCD screen, an LED screen, an OLED screen, etc.) incorporated into the electronic device 120. As another example, display 121 may include a removable display or projection system for providing display content on a surface remote from electronic device 120 (e.g., a video projector). The display 121 may be used to display content (e.g., information related to communication operations or information related to available media selections) under the direction of the control circuitry 126.
In one embodiment, input mechanism 124 may be any suitable mechanism or user interface for providing user input or instructions to electronic device 120. The input mechanism 124 may take a variety of forms, such as a button, keypad, dial, click wheel, mouse, visual indicator, remote control, one or more sensors (e.g., camera or visual sensor, light sensor, proximity sensor, etc.), or a touch screen. The input mechanism 124 may include a multi-touch display screen.
In one embodiment, the communication circuit 125 may be any suitable communication circuit that is operatively connected to a communication network (e.g., the
In some embodiments, the communication circuitry 125 may be used to create a communication network using any suitable communication protocol. For example, the communication circuit 125 may create a short-range communication network using a short-range communication protocol to connect to other communication devices. For example, the communication circuit 125 may be used to use
Protocol creates a local communication network to communicate with electronic device 120The earphones are coupled.In one embodiment, the control circuitry 126 may be used to control the actions and performance of the electronic device 120. Control circuitry 126 may include, for example, a processor, a bus (e.g., for sending instructions to other components of electronic device 120), a memory (memory), a storage device (storage), or any other suitable component for controlling actions of electronic device 120. In some embodiments, the processor may drive the display and process inputs received from the user interface. The memory and storage devices may include, for example, cache, flash, ROM, and/or RAM/DRAM. In some embodiments, the memory may be dedicated to storing firmware (e.g., for device applications such as an operating system, user interface functions, and processor functions). In some embodiments, the memory may be used to store information related to other devices with which the electronic device 120 performs communication operations (e.g., saving contact information related to communication operations or storing information related to different media types and media items selected by the user).
In one embodiment, the control circuitry 126 may be used to perform the actions of one or more applications implemented on the electronic device 120. Any suitable number or type of applications may be implemented. While the following discussion will list different applications, it should be understood that some or all of the applications may be combined into one or more applications. For example, electronic device 120 may include applications 1-N127, which include, but are not limited to: an Automatic Speech Recognition (ASR) application, an OCR application, a conversation application, a mapping application, a media application (e.g., QuickTime, Mobile music. app, or Mobile video. app), a social networking application (e.g., a social networking application
Etc.), calendar applications (e.g., calendars for managing events, appointments, etc.), internet browsing applications, etc. In some embodiments, electronic device 120 may include one or more applications operable to perform communication operations. For example, the electronic device 120 may include a messaging application, an email application, a voicemail application, an instant messaging application (e.g., for chat), a video conferencing application, a facsimile application, or any other suitable application to perform any suitable communication operations.In some embodiments, the electronic device 120 may include a microphone 122. For example, the electronic device 120 may include a microphone 122 to allow a user to send audio (e.g., voice audio) for voice control or navigation of applications 1-N127 during communication operations, or as an alternative to establishing communication operations or using a physical user interface. The microphone 122 may be incorporated into the electronic device 120 or may be remotely coupled to the electronic device 120. For example, the microphone 122 may be incorporated into a wired headset, the microphone 122 may be incorporated into a wireless headset, the microphone 122 may be incorporated into a remote control device, and so forth.
In one embodiment, the camera module 128 includes one or more camera devices including functionality for capturing still and video images, editing functionality, communications interoperability for sending, sharing photos/videos, and the like.
In one embodiment, electronic device 120 may include any other components suitable for performing communication operations. For example, the electronic device 120 may include a power source, port, or interface for coupling to a host device, an auxiliary input mechanism (e.g., an on/off switch), or any other suitable component.
FIG. 3 illustrates a rule
In some embodiments, the rule selector process has two sets of inputs,
c1={c1 d=At Home,c1 T=[2017:Nov:1:(12:23AM-11:05AM),2017:Nov:1:(7:02PM-8:04PM),...]}。
in some embodiments, the second input is a set of actions 320 (a). Each action aj ∈ A has a unique action descriptor aj d(e.g., view
Or charging a cell phone) and contains a series of time stamps a when the user initiates the actionj T. The rule selector handles the function of mining rules about thea3={a3 d=Begin Charging,a3 T=[2017:Nov:1:(10AM),2017:Nov:1:(5:45PM),...]}。
in some embodiments, based on the
In some embodiments, frequent context item set
f1={f1 d={At Home,Watch TV,Morning,Connected to Home Wi-Fi},fT=[2017:Nov:1:(9:03AM-9:40AM),2017:Nov:2:(8:45AM-9:34AM),...]}. To compute the frequent context item set, in one embodiment, the frequent context item set
In some embodiments, the reason for mining the frequent scenario item set F is that the frequent item set F e F acts as a precondition in the conditional action rules mined in the subsequent processing. In particular, for each action rule { c } output by the rule selector processing1 d,...,ck d}→{ad},{c1 d,...,ck dThe set of scene descriptors in F corresponds to the set of items F e F of descriptors fd of frequent events. By first mining a frequent scenario set of items, candidate conditional action rules are searched only in those rules whose precondition is a frequent set of items F e F, without considering all possible 2 s of scenario C|C|The subset serves as a potential precondition. Intuitively, if rule preconditions are not present in F, action rules will not be present often in the user's life, and thus are not worth selecting action rules for reminder or task automation.
In some embodiments, the rule selector process generates conditional action rules in a different manner than, for example, the pattern mining application process. In pattern mining applications, conditional action rules are derived by adding scenarios and actions in the input basket. This can create problems in calculating the confidence of conditional action rules because actions such as application usage are much less frequent than in long-term scenarios (e.g., at home). Instead, the rule selector process explicitly separates the scenario intervals and the user actions, and includes the scenario intervals only in the input basket. Then, conditional action rules are derived by searching candidate rules having a frequent set of scenario items as preconditions and actions as postconditions, and satisfying the rule selection metrics defined below. In some embodiments, the rule selector process generates a small set of fewer than seven conditional action rules for each action for selection by the end user. In contrast, conventional pattern mining processes produce hundreds of patterns per action and do not address the issue of rule aggregation or rule selection by the end user.
FIG. 4 illustrates an example table 400 of rule selection metrics for use in the rule selector process in accordance with some embodiments. Table 400 includes rule (selection)
In some embodiments, the
In some embodiments, the scenario-
In some embodiments, another important consideration for the user in selecting a rule is the number of unique instances at which the rule occurs. Interval count measurement from f when action a occursTThe number of unique time intervals of the subset. When the interval count is displayed to the user, it is normalized to the interval count per week. Again with respect to this metric, the variability preferred by the user can be found from the target action. For actions such as charging a smartphone, which is frequently performed by the user, the user prefers a higher interval count per week when selecting a rule. For actions such as ordering from a meal delivery service that the user performs only once or twice a week, the user may tolerate fewer interval counts per week.
In some embodiments, the
r1={In Cubicle,Between 5-6PM,Weekday,In Office,Battery Level<50%}→{Begin Charging}
r2={Weekday,Evening,Battery Level<50%}→{Begin Charging}
r3={At Home,Late Night}→{Begin Charging}
each of these rules covers a different subset of action instances than the set aT. For example, assume rule r1Covers 15 examples of charging the mobile phone by the user, rule r2Covers 22 uniqueAn instance of one; the total action range for these two rules is a union of two sets with 15 and 22 action instances, respectively. Since both rules are related to charging a cell phone at night, it can be seen that the union of the two rules covers only 23 unique instances of the user charging her cell phone. On the other hand, rules r2 and r3 cover a more diverse phone charging scheme, i.e., evening and midnight charging patterns at home; thus together these rules may cover an example of, for example, 40 users charging her cell phone. Thus, the total action scope of the rule set { r2, r3} is higher than the activity scope of the rule set { r1, r2 }. The scope of the action is a useful factor in the end user selection of the rule. Since the action scope enforces some diversity in the displayed rule set, the user has more opportunities to select various conditional action rules (e.g., collect reminders for family and work) and/or obtain more contextual precondition examples to define his own conditional rules.
FIG. 5 illustrates reduction and recommendation of rule selection according to some embodiments. In some embodiments, the rule selector process first eliminates most rules by thresholding the three rule (selection) metrics 410 (fig. 4), and then aggregates the rules to eliminate redundancy and maximize the scope of action. As shown in fig. 3, the purpose of
A key challenge in candidate rule generation is to determine a threshold parameter for each of the three rule selection metrics. In some embodiments, the rule selector user interface described below provides the end user with threshold parameters that control each metric. The reason behind this is that it may not be possible to set a single threshold for each that satisfies all user metrics; for example, some users may select rules with lower interval counts for initiating charging events, while other users may select rules with higher confidence and interval counts per week. Even for a single user, the user may select different threshold settings for different target actions. For example, the number of intervals between which the charging operation is started is high, and
the weekly interval count for the order of the action is low.In some embodiments, the goal of the rule summarization 360 (FIG. 3) process is to generate a small variety of conditional action rules for presentation to the user, with the goal of maximizing total action scope and eliminating rule redundancy.
Inputting: candidate conditional action rule set for action a R (a)
And (3) outputting: aggregated conditional action rule sets
1Set summarized rule set R(a)={}
while|R(a)|>0do
2select rule
such thatr=arg max
coverage(r∪R(a))if coverage(r∪R(a))=coverage(R(a))then
3break
4end
5add rule r to set
remove rule r from set R(a)
6end
7Output summarized rule set R(a).
Rule collections are another important part of the processing in the rule selector process. Rather than presenting the user with 50-100 candidate conditional action rules, they are aggregated into fewer than 10 to present a wide variety of conditional action rules that typically capture all of the action instances covered by the candidate rules. It is easier and more helpful for a user to browse through diverse, aggregated conditional action rules than a large number of redundant candidate rules. Furthermore, if one of the rules that the user actually desires is deleted as a redundant rule, the user can easily modify the closely related rule outputs to meet their needs through the rule selector process using the rule modification user interface discussed below.
In some embodiments, the rule selector process is implemented in the rule selector process application 129 (fig. 2) using a thin mobile client and cloud/server 130 architecture. The mobile scenario and motion data are recorded on the smartphone and periodically uploaded by the mobile device to the cloud/server 130 database. On the cloud/server 130, further scenario (including such things as Wi-Fi venues, important semantic venues, and activities such as commuting) inference is performed based on the recorded raw sensor data. The rule selector process then runs on the scenario/action data log on the cloud/server 130 to generate the aggregated conditional action rules. The following discusses how the rule exploration provides the user with user interface details for browsing, modifying and selecting conditional action rules. In some embodiments, the rule selector process may be implemented as a Java jar library, it being possible for the rule selector process to run entirely on mobile device 120 (FIG. 2).
To analyze the processing performance of the rule selector, mobile user scenarios and actions are collected. A list of collected movement actions and scenarios is listed below. Two different sets of mobile user data are used for the analysis. Based on the data collected from 200 selective smartphone users, the online, quantitative performance aspects of the rule selector process were each collected for 100 days. The data collection tool is used to collect the same scenarios and actions shown below for a smaller group of smartphone users. The list of collected scenes includes: the semantic place: home/work/significant location ID, Wi-Fi premises: at bay/Wi-Fi venue ID, time interval: time of day (morning, afternoon.. department), weekday/weekend, day of week, hours of day (0-23), battery charge: discretization into bins (bin) (0-25%, 25-50%, 0-50%, etc.), activity: SSID names obtained using the Android activity recognition API, at home and office commutes (as additional activities), and connected Wi-Fi networks. The list of collected actions and the list of user selection rules include application usage: application name started, call action: anonymous contact ID for call, SMS action: anonymous ID of the contact who sent the message, and start charging: and charging the mobile phone. In the above scenario and action list, it is noted that many scenarios and all actions may be recorded using the API. The following are three main scenarios for the rule selector to process the inference from the raw sensor data: Wi-Fi venues, important semantic venues, and commuting activities are discussed.
Using the Wi-Fi scan similarity metric, the entrances and exits of Wi-Fi premises are detected, and then each Wi-Fi premises visit is aggregated into a common set of an important set of Wi-Fi premises. To address some of the performance deficiencies of this approach, the joint Tanimoto Wi-Fi distance metric is replaced with a cross Tanimoto distance; the intersection distance metric calculates the Tanimoto distance only between common MAC addresses in two Wi-Fi scans, thereby reducing venue cluster (cluster) errors due to lack of MAC addresses. Likewise, idle activity inferred using the activity recognition API may enforce all Wi-Fi scans within an idle interval as part of the same Wi-Fi locale; the heuristic method further improves the accuracy of Wi-Fi place detection. For each Wi-Fi premises detected, the premises ID will be used as a scenario in the rule mining. The scenario is by marking the Wi-Fi premises, where the user spends the most time each day working at the cubicle, as a cubicle.
The detection of important places from streams of latitude/longitude location data is a widely studied subject of ubiquitous computing communities. In some embodiments, the algorithm detects location visits based on the GPS log and then aggregates the individual location visits into a significant cluster of locations. The rule selector process only retains clusters of locations where the user spends at least five minutes. Five minutes is chosen as a lower limit to detect potentially important locations (e.g., the morning coffee shop) where the user is only staying for a short time. Each non-empty distance cluster is assigned a unique important location ID and is used as a scenario for rule mining. To handle lost and outdated location data, the results of the Wi-Fi venue detection algorithm are fused with important semantic venue detection using GPS logs. Since most locations today have nearby Wi-Fi networks, this approach solves many data problems that would otherwise only use GPS data to corrupt important location detection. After gathering data to significant locations, home and office locations will be determined using heuristic based methods. To makeThe assumption used is that the location where users spend a lot of time in the evening is their home location, while the location where they spend a lot of time between 8 am and 6pm on a weekday is their office location. In addition, facts (Factual) are used for merchant labels that are available in semantics (e.g.Or
) Non-office locations and non-residential locations are marked.Using the important semantic place detection process described above, in some embodiments, inferences of when a user commutes are used. The commute detection is based on the following two pieces of information: 1) fromVehicle activity recognition data obtained by the activity recognition API, and 2) inferred home and office locations through significant semantic place detection. To identify commutes, a heuristic approach is used as follows: if the last observed location is home and it is the morning of a workday, then the activity is classified as commuting to work; if the last observed location was office and it was evening of a weekday, the activity is classified as commuting home.
In some embodiments, in the rule selector process, no single threshold is assigned to three separate rule selection metrics of rule confidence, context specificity, and interval count. Instead, a user is provided to select each parameter threshold using a user interface. Since it is not reasonable to expect a non-expert user to fully explore the space of possible parameter thresholds, in some embodiments, three convenient threshold settings (low, medium, and high) are provided for each of the three metrics, as shown in table I below. The low, medium, and high threshold settings are understood based on a preliminary discussion with the sample body under the requirements of the sample body and are selected based on empirical differences of the rule sets generated by the rule selector processing of each threshold setting. Rules that are too high or too low to generate any rules or to be useful to the end user are avoided.
[ TABLE 1 ]
In some embodiments, given the 3 parameters and threshold settings in table I, the user may explore 27 possible parameter combinations. The impact of these parameter thresholds is analyzed based on the number of candidate rules and the final aggregation rules generated by the rule selector processing. Table II shows the number of rules for three representative parameter settings: for the same sample user, all thresholds are set to low, medium and high, respectively. In the example analysis, it is noted that there are more than 2000 possible rules per action. In some embodiments, rule generation significantly reduces the rule selection metric threshold of interest to the smartphone user from possible rules to a set of high quality candidate rules. For the low and medium settings, it can be observed that the number of candidate rules is still too large (469 and 45, respectively) to be presented to the end user separately. In some embodiments, the rule selector process aggregates these candidate rules into fewer than ten rules for presentation to the user.
[ TABLE 2 ]
Table III provides an evaluation of how much the rule selector handles the action range maximization for the mid-set of all parameters in 200 sample data sets of smartphone users. On average, there are 20 actions per user with high quality candidate rules, and on average there are 66 candidate rules per operation. These candidate rules cover 74.5% of the user unique action instances. The rule selector processes all action instances covered by 66 rules can only be covered in 6-7 rules; sample users appreciate this highly because they can browse and select from a much smaller number of various conditional action rule sets.
[ TABLE 3 ]
Rule type
Number of regular actions
Number of rules per action
Total range of motion
Possible rules
119
226
100%
High quality candidate rules
20
66
74.5%
FIG. 6 illustrates a table 600 of example universal rules across populations, according to some embodiments. In table 600, the number of generic actions and generic rules is analyzed as the percentage of the user population sharing the actions and rules varies. Overall, it can be observed that there are few common actions and rules between sample users. For example, when viewing only 20% of the users, it can be observed that the common actions with high quality rules among the sample users drop significantly from 258 actions to only 11 actions. As a result, the number of common rules shared across users is also greatly reduced. Overall, only 4 rules were observed for more than 50% of users and only 15 rules for more than 30% of users.
FIG. 7 illustrates an
FIG. 8 illustrates an
FIG. 9 illustrates an
In some embodiments, the user is assigned different priorities to the various rule selection metrics depending on personal preferences and target actions. The user may select parameter settings depending on the operation and may modify the rules to adjust according to their needs and goals. In some embodiments, fine-grained scenarios may be used to improve the utility of the rules.
It should be noted that users often wish to apply rulesIn (i) reminders (e.g., charge reminders), (ii) recommendations (e.g., restaurant recommendations from a meal delivery service), and (iii) notifications (e.g., low priority notifications from a social media application). These applications help prevent negative user experience, such as low battery cell phones. In fact, the sample user always evaluates the battery charge reminder as being useful or very useful. In addition, users complain of notifications of how they eventually shut down certain applications, as these applications send overloaded messages at the wrong time in the event of a fault. More rule-based intelligent notification methods may limit notifications for certain applications to certain relevant scenarios, such as social media application notifications outside 8 am to 6pm on weekdays (P07) and providing only friday evenings and saturday evenings (P01)
It is proposed to solve this problem. Some embodiments provide meaningful shortcuts for important applications. For example, it is important to be able to access a map application while driving. Similarly, a user may wish to use a simple video application shortcut while at work, having lunch. A rule-based assistant should be designed around the three use cases described above, but still have flexibility to allow for future sets of use cases.FIG. 10 illustrates an example table 1000 of smart TV rules for an example user, according to some embodiments. In some embodiments, the rule selector process may be applied to a smart device such as a smart television system. Reminder rules may be established for favorite programs, sub-genres such as news, channels, etc., for example. Rules may be established for when recommendations for certain genres (e.g., movies, situation comedies, etc.) are displayed. The table 1000 includes
FIG. 11 illustrates an example table 1100 of scenario-driven rules for an example user, according to some embodiments. In some embodiments, the TV user interface may be personalized for different relevant scenarios. User actions may be predicted and action shortcuts may be provided in appropriate scenarios. Content (e.g., recommendations for situation comedies and movies) may be recommended based on predicted user operations. As shown, table 1100 includes example
Fig. 12A-12C illustrate example use cases of a smart device (e.g., electronic device 120) for rule setting, rule invocation, and personalized reminders, according to some embodiments. FIG. 12A shows an example display suggestion 1210 based on the following patterns: when the battery is less than 50%, users typically charge their devices in the office cubicles between 3-4 PM. The recommendation rule 1220 will be based on the usage pattern indicated in the suggestion 1210. In some embodiments, once the user accepts the suggested conditional action rule, the
FIG. 12B shows an
Fig. 12C shows a message and
Fig. 13A-13B illustrate
Fig. 14A-14C illustrate smart device 1400 (e.g., electronic device 120 of fig. 2) use cases for predicting presentation of user actions and contextual action cards/shortcuts, according to some embodiments. Fig. 14A shows a
Fig. 14B shows a
Fig. 14C shows a
In some embodiments, the rule selector process may be applied to other domains, such as common context data sets from both domains (smart home and smart tv), and clinical study data applied to improve mobility of elderly patients using smartphones. The rule selector process may generate personalized rules based on the smart-home sensor data. It is assumed that the information is collected from the residents R1 and R2. The scenario can be set to smart home user activity and room occupancy, and the actions can be set to light fixture usage, opening/closing doors, and using a particular cabinet of the kitchen. In one example, the kitchen cabinet 2 entry may indicate cooking and is used by resident R2 primarily during weekday afternoon when resident R2 sleeps. When resident R1 is not at home \ working or sleeping, the kitchen light may be used, which may strongly indicate that resident R2 seems to use the kitchen more for preparing meals. Example rule selector processing rules generated by smart home events may provide advanced functionality for smart devices. For example, knowing the rules about R2 times in the kitchen may help set when an intelligent device (e.g., an intelligent refrigerator) should inform family recipe suggestions and meal plan ideas. As another example, the user mode may be used to set an automatic schedule for the heating and lighting system. In some embodiments, the rule selector process determines personalized rules for both a single user and for multiple users, such as users in a family, a group of users (e.g., "all males between 20-25 years", etc.), or a general rule for the entire group of users. For each domain (e.g., smart home, medical, smart television, etc.), the rule selector process may select conditional action rules, so long as the data from each domain is represented as (i) a series of actions (e.g., "running of interest," "watch action movie," etc.), and (ii) each time each action is performed (e.g., "in the bedroom," "weather sunny," "friday evening," etc.).
Because users have different preferences for the rule selection parameters of actions, the real-time smartphone system must learn the preferences of users for parameters without giving too many rule suggestions to the users. In some embodiments, parameter learning for rule selection may be implemented and leverage techniques such as active learning or reinforcement learning are used. The user not only modifies the rules by deleting unnecessary conditions, but also adds new scenarios to the rules. Another aspect that may be implemented by some embodiments is to capture the order rule of the sequence of actions as opposed to co-occurring. For example, a user may use an application to find potential restaurants before posting a link to a chat. The sequence of checking applications and posting links continues until a consensus is reached for the restaurant. By learning this routine, the rule-based chat client can provide useful direct connectivity to the application, thereby making the entire user interaction more convenient. In general, sequential pattern mining is computationally a more resource-intensive co-occurrence mining, and also outputs a large number of automated candidate rules; these factors make sequence rule aggregation challenging. Some embodiments may implement periodic patterns of use, such as a monthly pattern for shopping at a particular store or website. Once the user accepts the rules, the rule selector process may present a UI that notifies the user of significant changes in mining patterns and prompts the user whether to update or maintain the rules. In an example embodiment, whenever a rule selected by a user is invalid, the user is notified because the rule no longer reflects the user's behavior. In this case, the user would be provided with suggestions to delete the rule or modify the rule based on the current user behavior. For example, the user selects a rule to remind her to call her parent on saturday evenings. User behavior may change over time and the user may ignore the reminder and frequently invoke their parents on weekday evenings. In such an example, the rule selector process notifies the user to suggest that she change the "call parents" reminder from saturday evening to sunday evening.
FIG. 15 illustrates a block diagram of a
In some embodiments, in
In some embodiments, in
FIG. 16 is a high-level block diagram illustrating an information handling system including a computing system implementing one or more embodiments. The system 1600 includes one or more processors 1611 (e.g., ASIC, CPU, etc.), and may further include an electronic display device 1612 (for displaying graphics, text, and other data), a main memory 1613 (e.g., Random Access Memory (RAM), cache device, etc.), a storage device 1614 (e.g., hard disk drive), a removable storage device 1615 (e.g., a removable storage drive, a removable memory, a magnetic tape drive, an optical disk drive, a computer readable medium having stored therein computer software and/or data), a user interface device 1616 (e.g., a keyboard, a touch screen, a keypad, a pointing device), and a communication interface 1617 (e.g., a modem, a wireless transceiver (e.g., Wi-Fi, cellular network)), a network interface (e.g., an ethernet card, a communications port, or a PCMCIA slot and card).
The communication interface 1617 allows software and data to be transferred between the computer system and external devices via the Internet 1650, the mobile electronic device 1651, the server 1652, the network 1653, and the like. The system 1600 also includes a communication infrastructure 1618 (e.g., a communication bus, crossbar, or network) to which the aforementioned devices 1611-1617 are connected.
Information conveyed via communications interface 1617 may be in the form of signals, such as electronic, electromagnetic, optical, or other signals capable of being received by communications interface 1617 via a communication link that carries signals, and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, a Radio Frequency (RF) link, and/or other communication channels.
In one implementation of one or more embodiments in a mobile wireless device (e.g., mobile phone, tablet, wearable device, etc.), the system 1600 also includes an image capture device 1620, such as the camera 128 (fig. 2), and an audio capture device 1619, such as the microphone 122 (fig. 2). The system 1600 may also include application processes or processors such as MMS 1621, SMS1622, email 1623, Social Network Interface (SNI)1624, audio/video (AV) player 1625, Web browser 1626, image capture 1627, and the like.
In one embodiment, system 1600 includes rule selector process 1630, which may implement a process similar to that described with respect to rule selector process application 129 (FIG. 2) and used for the processes described above with respect to FIGS. 5-10. In one embodiment, rule selector process 1630, along with operating system 1629, may be implemented as executable code residing in a memory of system 1600. In another embodiment, rule selector process 1630 may be provided in hardware, firmware, or the like.
In one embodiment, the main memory 1613, the storage device 1614, and the removable storage device 1615, each alone or in any combination, may store instructions of the above-described embodiments that may be executed by one or more processors 1611.
Those skilled in the art will appreciate that the foregoing example architectures described above, from which the embodiments can be implemented, can be implemented in many ways, e.g., as program instructions for execution by a processor, as software modules, microcode, as a computer program product on a computer readable medium, as analog/logic circuitry, as an application specific integrated circuit, as firmware, as consumer electronics, AV devices, wireless/wired transmitters, wireless/wired receivers, networks, multimedia devices, etc. Additionally, embodiments of the architecture may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements.
Embodiments have been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to one or more embodiments. Each block of such schematic/diagrams, or combinations thereof, may be implemented by computer program instructions. The computer program instructions, when provided to a processor, produce a machine, such that the instructions, which execute via the processor, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. Each block in the flow charts/block diagrams may represent a hardware and/or software module or logic to implement one or more embodiments. In alternative implementations, the functions noted in the block may occur out of the order noted in the figures, concurrently, or the like.
The terms "computer program medium," "computer usable medium," "computer readable medium," and "computer program product" are used to generally refer to media such as primary memory (main memory), secondary memory (secondary memory), removable storage drive, a hard disk installed in a hard disk drive, and the like. These computer program products are means for providing software to a computer system. The computer-readable medium allows the computer system to read data, instructions, messages or message packets, and other computer-readable information from the computer-readable medium. The computer-readable medium may include, for example, non-volatile memory, such as floppy disks, ROMs, flash memory, disk drive memory (disk drive memory), CD-ROMs, and other permanent storage devices. It is useful, for example, for transferring information, such as data and computer instructions, between computer systems. The computer program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The computer program instructions, which represent block diagrams and/or flowchart diagrams herein, may be loaded onto a computer, a programmable data processing apparatus, or a processing device such that the series of operations executed thereon produce a computer implemented process. Computer programs (i.e., computer control logic) are stored in the primary and/or secondary memories. The computer program may also be received via a communication interface. Such computer programs, when executed, cause a computer system to perform the features of one or more embodiments discussed herein. In particular, the computer programs, when executed, cause the processor and/or multi-core processor to perform the features of the computer system. Such computer programs represent controllers of the computer system. The computer program product includes a tangible storage medium readable by a computer system and storing instructions for execution by the computer system to perform the methods of one or more embodiments.
Although embodiments have been described with reference to certain versions thereof, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
- 上一篇:一种医用注射器针头装配设备
- 下一篇:推荐交通出行服务的系统和方法