阅读说明：本技术 智能音箱的播报控制方法和装置 (Broadcasting control method and device of intelligent sound box ) 是由 范冰冰 于 2020-04-23 设计创作，主要内容包括：本申请公开了一种智能音箱的播报控制方法和装置,涉及人工智能领域。具体实现方案为：获取预设区域的图像；判断图像之中是否包含用户的头像；以及如果包含用户的头像,则控制智能音箱进行语音播报。由此,当确定用户意图使用智能音箱时,才进行语音播报,避免语音播报被误触发时,对用户造成打扰,保证了在合适的实际进行语音播报,从而提升了智能音箱的服务质量。(The application discloses a broadcast control method and device of an intelligent sound box, and relates to the field of artificial intelligence. The specific implementation scheme is as follows: acquiring an image of a preset area; judging whether the image contains the head portrait of the user; and if the head portrait of the user is contained, controlling the intelligent sound box to broadcast the voice. From this, just carry out voice broadcast when confirming the user's intention and using intelligent audio amplifier, when avoiding voice broadcast to be triggered by the mistake, cause the disturbance to the user, guaranteed to carry out voice broadcast in suitable reality to intelligent audio amplifier's quality of service has been promoted.)

1. A broadcasting control method of an intelligent sound box is characterized by comprising the following steps:

acquiring an image of a preset area;

judging whether the image contains the head portrait of the user; and

if contain user's head portrait, then control intelligent audio amplifier carries out voice broadcast.

2. The announcement control method of a smart sound box according to claim 1, wherein the controlling the smart sound box to perform the voice announcement comprises:

acquiring age information and gender information of the user according to the head portrait of the user; and

and acquiring broadcasting recommended contents according to the age information and the gender information, and carrying out voice broadcasting according to the broadcasting recommended contents.

3. The announcement control method of an intelligent sound box according to claim 2, further comprising:

and acquiring current time, wherein broadcasting recommended content is acquired according to the age information, the gender information and the current time.

4. The announcement control method of an intelligent sound box according to claim 1, further comprising:

if the image does not contain the head portrait of the user, detecting whether the intelligent sound box is in a screen saver state;

and if the intelligent sound box is in the screen saver state, voice broadcasting is carried out when the screen saver state exits.

5. The announcement control method of claim 4, further comprising:

and if the intelligent sound box is not in the screen saver state, voice broadcasting is not carried out.

6. The announcement control method of a smart sound box according to claim 1, further comprising, during startup of the smart sound box:

detecting the starting state of the intelligent sound box;

if the starting state is an active starting state, controlling the intelligent sound box to carry out voice broadcasting;

and if the starting state is a passive starting state, controlling the intelligent sound box not to carry out voice broadcasting.

7. The announcement control method of claim 1, wherein the smart speaker has a display screen, and further comprising, before controlling the smart speaker to perform the voice announcement:

displaying a recommended function card on the display screen, wherein the function card corresponds to the voice broadcast content.

8. The utility model provides a control device that reports of intelligence audio amplifier which characterized in that includes:

the acquisition module is used for acquiring an image of a preset area;

the judging module is used for judging whether the image contains the head portrait of the user; and

and the control module is used for controlling the intelligent sound box to carry out voice broadcast when the head portrait of the user is contained.

9. The apparatus of claim 8, wherein the control module is specifically configured to:

acquiring age information and gender information of the user according to the head portrait of the user; and

and acquiring broadcasting recommended contents according to the age information and the gender information, and carrying out voice broadcasting according to the broadcasting recommended contents.

10. The apparatus of claim 8, wherein the apparatus further comprises:

the first detection module is used for detecting whether the intelligent sound box is in a screen saver state or not when the image does not contain the head portrait of the user;

the control module is further configured to:

and when the intelligent sound box is in the screen saver state, voice broadcasting is carried out when the screen saver state exits.

11. The apparatus of claim 8, wherein the apparatus further comprises:

the second detection module is used for detecting the starting state of the intelligent sound box;

the control module is further configured to:

when the starting state is an active starting state, controlling the intelligent sound box to carry out voice broadcasting;

and when the starting state is a passive starting state, controlling the intelligent sound box not to carry out voice broadcasting.

12. The apparatus of claim 8, wherein the smart speaker has a display screen, and wherein the control module is further configured to:

displaying a recommended function card on the display screen, wherein the function card corresponds to the voice broadcast content.

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 1-7.

14. A non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the announcement control method of a smart sound box according to any one of claims 1 to 7.

Technical Field

The present application relates to the field of artificial intelligence in the field of image processing technologies, and in particular, to a method and an apparatus for controlling broadcast of an intelligent speaker.

Background

Text-to-speech (TTS) technology is part of man-machine conversation and is widely used in smart speakers.

In the related art, when the smart speaker is powered on, the related voice broadcast is performed, however, the powering on of the smart speaker may be a false trigger deviating from the real intention of the user, for example, for a small smart speaker, as long as the user relates to a "small" keyword in a chat, the powering on of the small smart speaker will be performed, and the corresponding voice broadcast will be triggered by the false trigger powering on, so that the voice broadcast disturbs the user and affects the service quality of the smart speaker.

Disclosure of Invention

The application provides a broadcast control method and a broadcast control device for an intelligent sound box, wherein voice broadcast is performed when the intention of a user is determined, and the problem that the voice broadcast is disturbed when being triggered by mistake is avoided.

According to a first aspect, a broadcast control method for an intelligent sound box is provided, which includes: acquiring an image of a preset area; judging whether the image contains the head portrait of the user; and if contain user's head portrait, then control intelligent audio amplifier carries out voice broadcast.

According to a second aspect, a report controlling means of intelligent audio amplifier is provided, includes: the acquisition module is used for acquiring an image of a preset area; the judging module is used for judging whether the image contains the head portrait of the user; and the control module is used for controlling the intelligent sound box to carry out voice broadcast when the head portrait of the user is contained.

According to a third aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the broadcast control method of the smart sound box described in the above embodiment.

According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the announcement control method of a smart speaker described in the above embodiments.

The technical scheme provided by the embodiment of the application at least has the following beneficial technical effects:

when detecting to include the portrait in the image of predetermineeing the region, just control intelligent audio amplifier and carry out voice broadcast, from this, when confirming the user's intention and using intelligent audio amplifier, just carry out voice broadcast, when avoiding voice broadcast to be triggered by the mistake, cause the disturbance to the user, guaranteed to carry out voice broadcast in suitable reality to intelligent audio amplifier's quality of service has been promoted.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

fig. 1 is a schematic flow chart of a broadcast control method of a smart sound box according to a first embodiment of the present application;

fig. 2 is a schematic flow chart of a broadcast control method of a smart sound box according to a second embodiment of the present application;

fig. 3 is a schematic flow chart of a broadcast control method of a smart sound box according to a third embodiment of the present application;

FIG. 4 is a schematic diagram of a display interface of a smart sound box according to a fourth embodiment of the present application;

fig. 5 is a schematic view of a broadcast control scene of a smart sound box according to a fifth embodiment of the present application;

fig. 6 is a schematic structural diagram of a broadcast control device of a smart sound box according to a sixth embodiment of the present application;

fig. 7 is a schematic structural diagram of a broadcast control device of a smart sound box according to a seventh embodiment of the present application;

fig. 8 is a schematic structural diagram of a broadcast control device of a smart sound box according to an eighth embodiment of the present application;

fig. 9 is a block diagram of an electronic device for implementing a method for controlling broadcast of a smart sound box according to an embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

When carrying out the voice broadcast promptly to intelligent audio amplifier start, lead to probably that voice broadcast can disturb user's problem, this application provides one kind and holds the voice broadcast opportunity, carries out the mode of voice broadcast when the user needs.

Specifically, fig. 1 is a flowchart of a broadcast control method for a smart sound box according to an embodiment of the present application, and as shown in fig. 1, the method includes:

step 101, acquiring an image of a preset area.

The preset area may be an area in which an included angle with a screen is a preset range for a screen of the smart sound box, or an area in which a user is usually located when using the smart sound box, wherein the area in which the user is usually located may be a location where the user uses the history of the smart sound box within a preset time period, and when the number of times of the location of the history is greater than a preset threshold, the location of the history is considered to be the preset area.

Specifically, an image of the preset area is acquired, so that whether the user intends to use the smart sound box is determined according to the image content contained in the image. In practical application, the image of the preset area can be acquired according to the camera equipment in the intelligent loudspeaker box, and the intelligent equipment can be controlled to be networked with other equipment with cameras at home, so that the other equipment is controlled to acquire the image.

Step 102, judging whether the image contains the head portrait of the user.

Specifically, whether the image contains the head portrait of the user can be judged by extracting the image characteristics of the image and identifying whether the image characteristics contain the face characteristics; the contour information in the image can also be identified, and whether the image contains the head portrait of the user can be judged according to whether the extracted contour information contains the face contour.

In actual implementation, the avatar of the user may be the avatar of any user, or may be the avatar of a specific user set in advance.

And 103, if the head portrait of the user is contained, controlling the intelligent sound box to broadcast the voice.

It can be understood that, since the preset area generally corresponds to an area where the user uses the smart sound box, when the preset area includes the head portrait of the user, the user is considered to intend to use the smart sound box, and thus the smart sound box is controlled to perform voice announcement. Of course, when the image does not include the avatar of the user, in order to further improve the service quality, it may be determined whether the time for actively performing the voice broadcast is satisfied.

In an embodiment of the present application, in consideration of practical applications, if the smart speaker is in the screen saver state, it is considered that the user has used the smart speaker in the near future, and therefore, in order to avoid the situation that the user is unaware of the fact that the smart speaker directly enters the sleep state, in this embodiment, it is also possible to detect whether the smart speaker is in the screen saver state, for example, whether a current display image of the smart speaker is a preset screen saver image, for example, a current running program interface of the smart speaker is detected, and whether the screen saver application is currently run is determined according to the interface.

Further, if the intelligent sound box is in the screen saver state, voice broadcasting is carried out to inform the user when the screen saver state exits. Such as informing the user "i am sleeping and cheerful owner".

If the intelligent sound box is not in the screen saver state, the intelligent sound box is determined not to meet the condition of active voice broadcasting, and therefore voice broadcasting is not carried out.

In another embodiment of the present application, even if the portrait of the user is not collected in the preset area, the user may actively turn on the smart speaker, and in this case, the condition of voice broadcast is obviously satisfied.

Specifically, the start state of the smart sound box is detected, for example, the start state may be determined according to a current upper layer interface calling condition of the smart sound box, or the start state may be determined according to a triggered object, if the start state is an active start state, for example, an object called by an upper layer interface is a start key, or the start state is a power access state, for example, a restart key is triggered, the smart sound box is controlled to perform voice broadcast, and if the start state is a passive start state, for example, an automatic start-up state, or a restart silence state, the smart sound box is controlled not to perform voice broadcast.

Therefore, when the smart sound box broadcasts in this embodiment, the real intention of the user is considered from multiple dimensions, and whether to broadcast the sound is determined according to the real intention of the user, so that those skilled in the art can understand the application more clearly.

In this scenario, the image of the preset area is collected by the camera in the smart speaker, as shown in fig. 2, it can be detected whether the camera of the smart speaker supports face detection, if not, voice broadcast is not performed, wherein a message reminding the smart speaker of version upgrade can be sent to the user, so as to facilitate the implementation of face detection subsequently.

If support, whether the camera is opened or not is judged, if the camera is opened, whether a face is contained in an image of a preset area or not is obtained and detected, if the face is contained, voice broadcasting is carried out, if the face is not contained, whether the intelligent sound box is in a screen saver state or not is detected, if the intelligent sound box is in a screen saver state, voice broadcasting is carried out after the screen saver state exits, wherein the screen saver state exits automatically after the preset time is reached, and the screen saver state also can be exited by triggering a corresponding control by a user. If the intelligent sound box is not in the screen saver state, whether the portrait is recognized or not is waited for, and meanwhile broadcasting is not carried out.

If the camera of intelligent audio amplifier is not opened, then detect the start state of current intelligent audio amplifier, if the start state is initiative start-up state, then control intelligent audio amplifier and carry out voice broadcast, if the start state is passive start-up state, then control intelligent audio amplifier not to carry out voice broadcast.

To sum up, the broadcast control method of the intelligent sound box in the embodiment of the application controls the intelligent sound box to perform voice broadcast when detecting that the image of the preset area contains the portrait, so that the voice broadcast is performed when determining that the user intends to use the intelligent sound box, the voice broadcast is avoided from being disturbed when being triggered by mistake, the voice broadcast is actually performed properly, and therefore the service quality of the intelligent sound box is improved.

Based on the above embodiment, in order to further improve the service quality of the smart sound box, the voice broadcast mode can be flexibly determined according to the current scene needs.

In one embodiment of the present application, as shown in fig. 3, the step 103 includes:

step 201, acquiring age information and gender information of a user according to the head portrait of the user.

It is easily understood that the avatar of the user reflects the age information and the gender information of the user, and thus, the age information and the gender information of the user can be acquired according to the avatar of the user.

It should be noted that, in different application scenarios, acquiring age information and gender information of a user according to an avatar of the user may be implemented in different manners:

in an embodiment of the present application, portrait features corresponding to different age information and gender information may be pre-constructed, for example, portrait features corresponding to women include a long hair feature, a makeup feature, and the like, and portrait features corresponding to men include a beard feature, and the like, so as to extract a portrait feature of a user's head, compare the portrait feature with the pre-constructed portrait feature, and determine the age information and the gender information of the user according to a comparison result.

In another embodiment of the present application, a deep learning model may be obtained by training in advance according to a large number of sample images, and the deep learning model inputs an image including a portrait area and outputs age information and gender information.

And step 202, obtaining the broadcast recommended content according to the age information and the gender information, and carrying out voice broadcast according to the broadcast recommended content.

Specifically, because the contents of interest of different ages and sexes are different, for example, middle-aged men generally have interest in financial information, the broadcast recommended content is acquired according to the age information and the gender information, for example, a preset corresponding relationship can be queried, the broadcast recommended content corresponding to the age information and the gender is determined, and then, voice broadcast is performed according to the broadcast recommended content.

Of course, in order to further avoid disturbing the user, the broadcast time may be selected when performing voice broadcast, and in some possible examples, the voice broadcast is performed for the user at a suitable time, for example, the historical use time of the user using the sound box is counted according to the portrait information of the user, the current time is obtained, and the voice broadcast is performed when the current time is matched with the historical use time;

in other possible examples, broadcast time history data of a large number of users corresponding to broadcast recommended content may be determined, the most frequently broadcast time of the users is determined according to the broadcast time history data, the current time is obtained, and voice broadcast is performed when the current time matches the most frequently broadcast time of the users.

In another embodiment of the present application, as shown in fig. 4, the smart sound box has a display screen, and therefore before the smart sound box is manufactured to perform the voice broadcasting, a recommended function card can be displayed on the display screen, wherein the function card corresponds to the voice broadcasting content, and further, when the voice broadcasting is performed, the user can also visually see the corresponding voice broadcasting content.

In another embodiment of the present application, in order to further improve the user experience of the user, the age information of the user may be determined according to the portrait information of the user, and the preset database is queried according to the age information of the user to obtain the matched broadcast sound information and volume information, for example, as shown in fig. 5, when the age of the user is 10 years old, volume voice broadcast in cartoon sound is adopted, and when the age of the user is 60 years old, volume voice broadcast in a sound cavity is adopted.

In summary, the broadcasting control method for the intelligent sound box according to the embodiment of the application can flexibly select the voice broadcasting mode according to the actual scene, and further improves the service quality of the intelligent sound box.

In order to implement the foregoing embodiment, this application further provides a broadcast control device of a smart speaker, and fig. 6 is a schematic structural diagram of a broadcast control device of a smart speaker according to an embodiment of this application, as shown in fig. 6, the broadcast control device of a smart speaker includes: an obtaining module 61, a judging module 62, and a control module 63, wherein,

an obtaining module 61, configured to obtain an image of a preset region;

specifically, the obtaining module 61 obtains an image of the preset area, so as to determine whether the user intends to use the smart speaker according to the image content contained in the image. In practical application, the image of the preset area can be acquired according to the camera equipment in the intelligent loudspeaker box, and the intelligent equipment can be controlled to be networked with other equipment with cameras at home, so that the other equipment is controlled to acquire the image.

A judging module 62, configured to judge whether the image includes a head portrait of the user;

specifically, the judging module 62 may judge whether the image includes the head portrait of the user by extracting image features of the image and identifying whether the image features include face features; the contour information in the image can also be identified, and whether the image contains the head portrait of the user can be judged according to whether the extracted contour information contains the face contour.

And the control module 63 is used for controlling the intelligent sound box to perform voice broadcasting when the head portrait of the user is included.

In an embodiment of the present application, in consideration of practical applications, if the smart speaker is in the screen saver state, it is considered that the user has used the smart speaker in the near future, and therefore, in order to avoid the situation that the user is unaware of the fact that the smart speaker directly enters the sleep state, in this embodiment, it is also possible to detect whether the smart speaker is in the screen saver state, for example, whether a current display image of the smart speaker is a preset screen saver image, for example, a current running program interface of the smart speaker is detected, and whether the screen saver application is currently run is determined according to the interface.

Specifically, as shown in fig. 7, the apparatus further includes, on the basis of fig. 6: a first detection module 64 that, among other things,

the first detection module 64 is configured to detect whether the smart speaker is in a screen saver state when the image does not include the head portrait of the user;

in this embodiment, the control module 63 is further configured to:

when the intelligent sound box is in the screen saver state, voice broadcasting is carried out when the screen saver state exits.

In an embodiment of the application, even if the portrait of the user is not collected in the preset area, the user may actively turn on the smart speaker, and in this case, the condition of voice broadcast is obviously satisfied.

As shown in fig. 8, on the basis of that shown in fig. 6, the apparatus further includes: a second detection module 65, wherein,

the second detection module 65 is configured to detect a start state of the smart speaker;

the control module 63, further configured to:

when the starting state is the active starting state, controlling the intelligent sound box to broadcast the voice;

and when the starting state is the passive starting state, controlling the intelligent sound box not to broadcast the voice.

Specifically, the second detection module 65 detects a start state of the smart speaker, for example, the second detection module 65 may determine the start state according to a current upper layer interface calling condition of the smart speaker, or may determine the start state according to a triggered object, if the start state is an active start state, for example, an object called by the upper layer interface is a start key, or the start state is a power access state, or for example, the start state is a restart key triggered, the control module 63 controls the smart speaker to perform voice broadcast, and if the start state is a passive start state, for example, an automatic start state, or a restart silent state, the smart speaker is controlled not to perform voice broadcast.

It should be noted that, the foregoing explanation of the broadcast control method for the smart sound box also makes the broadcast control device applicable to the smart sound box of this embodiment, and the implementation principle thereof is similar, and is not repeated herein.

To sum up, the intelligent audio amplifier's of this application embodiment report controlling means, when detecting to include the portrait in the image of predetermineeing the region, just control intelligent audio amplifier and carry out voice broadcast, from this, when confirming that the user intends to use intelligent audio amplifier, just carry out voice broadcast, when avoiding voice broadcast to be triggered by the mistake, cause the disturbance to the user, guaranteed to carry out voice broadcast in suitable reality to the quality of service of intelligent audio amplifier has been promoted.

Based on the above embodiment, in order to further improve the service quality of the smart sound box, the voice broadcast mode can be flexibly determined according to the current scene needs.

In an embodiment of the present application, the control module 63 is specifically configured to:

acquiring age information and gender information of a user according to the head portrait of the user; and

and acquiring the broadcast recommended content according to the age information and the gender information, and carrying out voice broadcast according to the broadcast recommended content.

In an embodiment of the present application, the smart speaker has a display screen, and the control module 63 is further configured to:

and displaying the recommended function card on the display screen, wherein the function card corresponds to the voice broadcast content.

To sum up, the intelligent sound box broadcast control device can flexibly select the voice broadcast mode according to the actual scene, and further improves the service quality of the intelligent sound box.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 9 is a block diagram of an electronic device according to an embodiment of the present application, where the method is used for controlling broadcast of a smart sound box. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 9, the electronic apparatus includes: one or more processors 901, memory 902, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 9 illustrates an example of a processor 901.

Memory 902 is a non-transitory computer readable storage medium as provided herein. The memory stores instructions executable by at least one processor, so that the at least one processor executes the method for controlling the broadcast of the smart sound box provided by the application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of announcement control of a smart sound box provided by the present application.

The memory 902, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method for controlling playback of a smart sound box in the embodiment of the present application (for example, the obtaining module 61, the determining module 62, and the control module 63 shown in fig. 6). The processor 901 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 902, that is, implementing the method for controlling broadcast of the smart speaker in the above method embodiment.

The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device controlled by the broadcast of the smart speaker, and the like. Further, the memory 902 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 902 may optionally include memory located remotely from processor 901, which may be connected to the announcement controlled electronics of the smart speaker via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the broadcast control method of the smart speaker may further include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903 and the output device 904 may be connected by a bus or other means, and fig. 9 illustrates the connection by a bus as an example.

The input device 903 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of the smart speaker's broadcast controlled electronic device, such as a touch screen, keypad, mouse, track pad, touch pad, pointer stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 904 may include a display device, auxiliary lighting devices (e.g., LEDs), tactile feedback devices (e.g., vibrating motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.