阅读说明：本技术 推荐方法及装置 (Recommendation method and device ) 是由 程飞 于 2018-09-07 设计创作，主要内容包括：本说明书一个或多个实施例提供一种推荐方法及装置,该方法可以包括：获取用户图像；采集用户语音；根据所述用户语音确定目标图像；叠加展示所述用户图像和所述目标图像。(One or more embodiments of the present specification provide a recommendation method and apparatus, where the method may include: acquiring a user image; collecting user voice; determining a target image according to the user voice; and displaying the user image and the target image in an overlapping manner.)

1. A recommendation method, comprising:

acquiring a user image;

collecting user voice;

determining a target image according to the user voice;

and displaying the user image and the target image in an overlapping manner.

2. The method of claim 1, wherein the obtaining the user image comprises:

acquiring an image of the user to obtain an image of the user;

alternatively, the image data provided by the user is acquired as the user image.

3. The method of claim 1, wherein determining a target image from the user speech comprises:

analyzing and recognizing the user voice locally or under the assistance of a cloud end to determine the requirement information of the user;

and determining a target image which accords with the requirement information.

4. The method of claim 1, wherein determining a target image from the user speech comprises:

displaying alternative requirements;

determining the selected alternative requirements as requirement information of the user according to the selection result of the user voice on the alternative requirements;

and determining a target image which accords with the requirement information.

5. The method according to claim 3 or 4, wherein the determining of the target image that meets the requirement information comprises:

and determining a target image meeting the requirement information locally or under the assistance of a cloud.

6. The method of claim 3 or 4, wherein the demand information comprises at least one of: is suitable for occasions and personal preferences.

7. The method of claim 1, further comprising:

extracting user characteristic information contained in the user image;

wherein the target image further matches the user characteristic information.

8. The method according to claim 7, wherein the extracting user feature information included in the user image comprises:

analyzing the user image locally or with the assistance of a cloud to obtain the user characteristic information.

9. The method of claim 7, wherein the user characteristic information comprises at least one of: age, face shape, position of five sense organs, skin color, hair style, hair color, and wearing.

10. The method of claim 1, wherein the target image comprises at least one of:

makeup images, hairstyle images, headwear images, and wearing images.

11. The method of claim 1, further comprising:

and providing a downloading channel or a sharing channel of the target image or the user image superposed with the target image for the user.

12. The method of claim 1, further comprising:

and sending a printing instruction to a built-in printing module or a peripheral related printing device to print the target image or the user image superposed with the target image.

13. The method of claim 1, further comprising:

interacting with the user a commodity for realizing the presentation effect of the target image, a selling shop of the commodity and/or a shop providing a service related to the target image.

14. The method of claim 1, further comprising:

determining preference information of the user for the target image;

and uploading the preference information to a cloud end, so that recommended contents related to the preference information are pushed to the user by the cloud end.

15. The method of claim 1, further comprising:

performing authority verification on the user;

and when the user has a preset right, allowing the user image and the target image to be displayed to the user in an overlapping mode.

16. A recommendation device, comprising:

an image acquisition unit that acquires a user image;

the voice acquisition unit is used for acquiring user voice;

an image determining unit that determines a target image from the user voice;

and the superposition display unit is used for displaying the user image and the target image in a superposition manner.

17. The apparatus according to claim 16, wherein the image acquisition unit is specifically configured to:

acquiring an image of the user to obtain an image of the user;

alternatively, the image data provided by the user is acquired as the user image.

18. The apparatus according to claim 16, wherein the image determination unit is specifically configured to:

analyzing and recognizing the user voice locally or under the assistance of a cloud end to determine the requirement information of the user;

and determining a target image which accords with the requirement information.

19. The apparatus according to claim 16, wherein the image determination unit is specifically configured to:

displaying alternative requirements;

determining the selected alternative requirements as requirement information of the user according to the selection result of the user voice on the alternative requirements;

and determining a target image which accords with the requirement information.

20. The apparatus according to claim 18 or 19, wherein the image determining unit determines the target image that meets the requirement information by:

and determining a target image meeting the requirement information locally or under the assistance of a cloud.

21. The apparatus of claim 18 or 19, wherein the requirement information comprises at least one of: is suitable for occasions and personal preferences.

22. The apparatus of claim 16, further comprising:

a feature extraction unit that extracts user feature information included in the user image;

wherein the target image further matches the user characteristic information.

23. The apparatus according to claim 22, wherein the feature extraction unit is specifically configured to:

analyzing the user image locally or with the assistance of a cloud to obtain the user characteristic information.

24. The apparatus of claim 22, wherein the user characteristic information comprises at least one of: age, face shape, position of five sense organs, skin color, hair style, hair color, and wearing.

25. The apparatus of claim 16, wherein the target image comprises at least one of:

makeup images, hairstyle images, headwear images, and wearing images.

26. The apparatus of claim 16, further comprising:

and the channel providing unit is used for providing a downloading channel or a sharing channel of the target image or the user image superposed with the target image for the user.

27. The apparatus of claim 16, further comprising:

and the instruction sending unit is used for sending a printing instruction to a built-in printing module or peripheral associated printing equipment so as to print the target image or the user image superposed with the target image.

28. The apparatus of claim 16, further comprising:

and an information interaction unit which interacts with the user for a commodity for realizing the presentation effect of the target image, a sales shop of the commodity, and/or a shop providing a service related to the target image.

29. The apparatus of claim 16, further comprising:

a preference determining unit that determines preference information of the user for the target image;

and the preference uploading unit uploads the preference information to a cloud end so as to push recommended content related to the preference information to the user by the cloud end.

30. The apparatus of claim 16, further comprising:

the authority verification unit is used for verifying the authority of the user;

and the interaction control unit allows the user image and the target image to be displayed in an overlapping mode to the user when the user has the preset authority.

Technical Field

One or more embodiments of the present disclosure relate to the field of smart device technologies, and in particular, to a recommendation method and apparatus.

Background

By proper decoration, such as makeup, hairstyle change, clothing matching and the like, not only can the visual effect be improved, but also the mental aspect of the user can be improved. In the related art, users with rich dressing experience can share various dressing schemes finished by the users to the network, and other users can check and select the schemes on the network to determine the dressing schemes which are possibly suitable for the users.

Disclosure of Invention

In view of this, one or more embodiments of the present disclosure provide a recommendation method and apparatus.

To achieve the above object, one or more embodiments of the present disclosure provide the following technical solutions:

according to a first aspect of one or more embodiments of the present specification, there is provided a recommendation method including:

acquiring a user image;

collecting user voice;

determining a target image according to the user voice;

and displaying the user image and the target image in an overlapping manner.

According to a second aspect of one or more embodiments of the present specification, there is provided a recommendation apparatus including:

an image acquisition unit that acquires a user image;

the voice acquisition unit is used for acquiring user voice;

an image determining unit that determines a target image from the user voice;

and the superposition display unit is used for displaying the user image and the target image in a superposition manner.

Drawings

Fig. 1 is a schematic diagram of an architecture of a recommendation system according to an exemplary embodiment.

FIG. 2 is a flow chart of a recommendation method provided by an exemplary embodiment.

Fig. 3 is a schematic diagram of a scene for recommending makeup to a user according to an exemplary embodiment.

FIG. 4 is a schematic diagram of guiding a user for voice interaction according to an exemplary embodiment.

Fig. 5 is a schematic diagram of an intelligent mirror device locally recommending makeup to a user according to an exemplary embodiment.

Fig. 6 is a schematic diagram illustrating that a smart mirror device recommends makeup to a user with the assistance of a cloud server according to an exemplary embodiment.

FIG. 7 is a schematic diagram of printing a user image after makeup according to an exemplary embodiment.

FIG. 8 is a schematic diagram of delivering interactive advertisements, according to an example embodiment.

Fig. 9 is a schematic structural diagram of an apparatus according to an exemplary embodiment.

Fig. 10 is a block diagram of a recommendation device according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of one or more embodiments of the specification, as detailed in the claims which follow.

It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described herein. In some other embodiments, the method may include more or fewer steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.

In an embodiment, the recommendation scheme of the present specification may be applied to an electronic device, for example, the electronic device may include any type of mobile phone, tablet device, notebook computer, Personal Digital Assistants (PDAs), wearable device (e.g., smart glasses, smart watch, etc.), outdoor sharing device (e.g., smart mirror device, smart billboard, etc.), etc., and the present specification does not limit this. In the operation process, the electronic equipment can recommend a proper target image to the user through man-machine interaction with the user.

Fig. 1 is a schematic diagram of an architecture of a recommendation system according to an exemplary embodiment. As shown in fig. 1, the system may comprise a server 11, a network 12, a number of electronic devices, such as at a smart mirror device 13, a smart mirror device 14, etc.

The server 11 may be a physical server comprising a separate host, or the server 11 may be a virtual server carried by a cluster of hosts. During the operation, the server 11 may operate a program on a server side of the recommended application to be implemented as a server side of the recommended application. In one or more embodiments of the present disclosure, the server 11 may cooperate with the clients running on the smart mirror devices 13-14 to implement the recommended solution.

The smart mirror devices 13-14 are only one type of electronic device that can be used. In practice it is obvious that also electronic devices of the type such as: tablet devices, notebook computers, Personal Digital Assistants (PDAs), wearable devices (e.g., smart glasses, smart watches, etc.), outdoor sharing devices (e.g., smart mirror devices, smart billboards, etc.), etc., which are not limited by one or more embodiments of the present disclosure. During operation, the electronic device may run a program on a client side of the recommended application to be implemented as a client of the recommended application.

It should be noted that: an application program of a client recommending an application can be pre-installed on the electronic device, so that the client can be started and run on the electronic device; of course, when an online "client" such as HTML5 technology is employed, the client can be obtained and run without installing a corresponding application on the electronic device.

While the network 12 for interaction between the smart mirror devices 13-14 and the server 11 may comprise various types of wired or wireless networks. In one embodiment, the Network 12 may include the Public Switched Telephone Network (PSTN) and the Internet.

FIG. 2 is a flow chart of a recommendation method provided by an exemplary embodiment. As shown in fig. 2, the method applied to an electronic device (e.g., the smart mirror devices 13-14 shown in fig. 1, etc.) may include the following steps:

step 202, acquiring a user image.

In an embodiment, a camera is disposed on the electronic device, and the user may be subjected to image acquisition by the camera to obtain the user image. Wherein the user image may comprise a plurality of forms; for example, the user image may be in the form of a picture, that is, the camera may take pictures of the user to obtain one or more pictures; for another example, the user image may be in a video form, that is, the camera may take a video of the user, and the taken video may include a plurality of image frames, which is equivalent to taking a plurality of pictures of the user.

In an embodiment, the camera on the electronic device may be of any type, and this specification does not limit this. For example, the camera may be a common RGB camera; for another example, the camera may be a depth camera, and may better perform processing such as background blurring, background matting, user feature recognition, and the like on the user image. The number of the cameras may be single or multiple (for example, a binocular structure formed by two cameras, etc.), and this specification does not limit this.

In an embodiment, the image data provided by the user can be acquired as the user image regardless of whether a camera is provided on the electronic device. For example, a user may establish a local area network connection, such as a bluetooth connection, a WIFI connection, and the like, between a user device, such as a mobile phone and a notebook computer, and transmit image data (such as pictures or videos) on the user device to the electronic device based on the local area network connection; for another example, the user may upload image data in the user equipment to a cloud, and push the image data to the electronic equipment from the cloud; for another example, the user may log in his own account on the electronic device, so that the electronic device may obtain the image data from a cloud disk, a social network site, and the like corresponding to the account; the electronic device may also obtain the image data of the user in other manners, which are not listed here, and this description does not limit this.

Step 204, collecting the user voice.

In one embodiment, a microphone module may be disposed on the electronic device for collecting voice of the user. The microphone module can adopt a microphone array structure, so that the electronic equipment can distinguish voice sources, accurately distinguish voices, emitted by users, corresponding to the collected user images, and exclude voices, background noises and the like emitted by other users.

Step 206, determining a target image according to the user voice.

In one embodiment, a user can express own requirement information through a user voice, so as to determine a target image which conforms to the requirement information.

In one embodiment, the electronic device may perform analysis and recognition on the user voice locally to obtain the requirement information. For example, a voice processing chip (such as a voice AI chip) may be built in the electronic device, and the voice of the user can be analyzed and recognized locally and quickly, so as to determine the requirement information of the user.

In an embodiment, the electronic device may analyze and recognize the user voice under the assistance of a cloud to obtain the requirement information. For example, the electronic device may upload the user voice to the cloud, analyze and recognize the user voice by the cloud, and receive a recognition result returned by the cloud, thereby determining the demand information.

In one embodiment, the voice content contained in the user voice may directly contain the demand content of the demand information. For example, when the user voice is "what make up i want to go to date", it is indicated that the user's demand information is "make up suitable for date", that is, the demand information of the user can be directly determined only by recognizing the voice content included in the user voice.

In an embodiment, the electronic device may show alternative requirements to the user through the display screen, for example, alternative requirement 1 is "what make up should be taken by me for appointment, alternative requirement 2 is" what make up should be taken by me for meeting ", alternative requirement 3 is" what make up is popular recently ", and the like, so that the user can view and select the alternative requirements; correspondingly, the voice content contained in the user voice is used for expressing the selection result of the alternative demands, and the electronic equipment can determine the selected alternative demands as the demand information according to the selection result of the user voice on the alternative demands. For example, when the user voice is "first", it may be determined that the selection result is that the above-described alternative requirement 1 is selected, and thus the requirement information of the user is "makeup appropriate for an appointment".

In one embodiment, the requirement information of the user may include at least one of: the method is suitable for occasions (such as lovers' appointments, friends meeting, company meeting, traveling out, etc.), personal preferences (such as shade degree, style preference, etc.), etc., and the description does not limit the same; in fact, any information capable of expressing the user's needs can be used as the need information here.

In one embodiment, the electronic device may determine the recommended makeup locally and interact with the user. For example, the electronic device may include an image processing chip (e.g., an AI chip, etc.) so that the electronic device may respond quickly to the user, interacting with the recommended makeup as described above.

In one embodiment, the electronic device may determine the recommended makeup with assistance from the cloud for interaction with the user. For example, the electronic device may upload the demand information of the user to the cloud (or determine the demand information of the user after analyzing and recognizing the voice of the user by the cloud), and determine the recommended makeup by the cloud, and the electronic device may receive the recommended makeup returned by the cloud to interact with the user.

And step 208, displaying the user image and the target image in an overlapping mode.

In one embodiment, the target image in the present specification may include at least one of: a makeup image (for presenting a virtual makeup effect), a hairstyle image (for presenting a virtual hairstyle effect), a headwear image (for presenting a virtual head ornament wearing effect), a wearing image (for presenting a virtual clothing wearing effect), and the like, which are not limited in the present specification.

In an embodiment, the electronic device may present the user image and the target image to the user in an overlay. For example, when the user image is a picture, the picture may be processed so that the target image is superimposed on the picture (for example, when the target image is a makeup image, the target image may be superimposed on a face shown in the picture), so that the user can accordingly know a state of the user (the user may be the user himself or any other user) in the picture when the target image is adopted, and determine whether makeup, hair style, headwear, wearing and the like corresponding to the target image need to be adopted. For another example, the user image may be a real-time image taken by the user, that is, after the user is subjected to image acquisition by a camera on the electronic device, the user is displayed on the display screen of the electronic device without any delay, so that the user looks like looking into a mirror when viewing the user image on the display screen; correspondingly, the electronic equipment displays the target image on the user image in a superposed manner by taking the target image as a makeup image, so that the user looks like watching the user who has made up the makeup, and can more accurately know the condition of the user after the user uses the recommended makeup; particularly, when the camera on the electronic device is a depth camera, the target image displayed in an overlapping manner can be made to more closely fit the limbs of the user (for example, the recommended makeup is made to more closely fit the face of the user) based on the depth information, and corresponding changes can be realized along with the distance adjustment, the angle adjustment and the like between the user and the electronic device, so that the user can more carefully and accurately know the conditions after using the makeup, the hair style, the head gear, the putting on and the like corresponding to the target image.

In an embodiment, the electronic device may interact the target image with the user in one or more other ways besides the overlay presentation described above, which is not limited by this specification. For example, the electronic device may transmit the target image to a user device used by the user, such as near field transmission over a local area network connection or remote transmission over a cloud.

In one embodiment, the electronic device may extract user feature information included in the user image; and the target image is also matched with the user characteristic information, so that the target image is more in line with the actual situation of the user. In other words, the electronic device may interact with the user a target image that matches the requirement information and matches the user feature information. For example, the user characteristic information may include at least one of: age, facial form, position of five sense organs, skin tone, hair style, hair color, wearing arrangement, etc., which the present specification does not limit.

In one embodiment, the electronic device may analyze the user image locally to obtain the user characteristic information. For example, the electronic device may have an image processing chip built therein, and may analyze user feature information included in the user image.

In an embodiment, the electronic device may analyze the user image with assistance of a cloud to obtain the user feature information. For example, the electronic device may upload the user image to the cloud, and receive the user feature information returned after the analysis by the cloud.

In an embodiment, the electronic device may provide a download channel for the target image or the user image superimposed with the target image to the user, so that the user may download the target image or the user image superimposed with the target image to a local storage space of the user device, a storage space of the intelligent router, a cloud disk storage space, or the like through the download channel.

In an embodiment, the electronic device may provide a sharing channel for the target image or the user image after the target image is superimposed to the user, so that the user may share the target image or the user image after the target image is superimposed to another user or share the user image to a social platform of the user through the sharing channel.

In an embodiment, the electronic device may send a print instruction to a built-in print module to print the target image or the user image on which the target image is superimposed. Alternatively, the electronic device may be connected (wired or wirelessly) to an associated printing device of the peripheral device, so that the electronic device may print the target image or the user image on which the target image is superimposed by the associated printing device by transmitting a print instruction to the associated printing device. Of course, the electronic device should also provide the target image to be printed or the user image superimposed with the target image to the built-in printing module or the associated printing device; for example, the built-in printing module may read a target image or a user image obtained by superimposing the target image from a memory of the electronic device; for another example, the electronic device may directly send the target image or the user image on which the target image is superimposed to the associated printing device, or the electronic device may provide a download address of the target image or the user image on which the target image is superimposed to the associated printing device, so that the associated printing device may obtain the target image or the user image on which the target image is superimposed from the download address.

In one embodiment, the electronic device may interact with the user a commodity for realizing the presentation effect of the target image, a sales shop of the commodity, and/or a shop providing a service related to the target image, so that the user purchases the related commodity or goes to the related shop to experience the related service. Taking a target image as a makeup image as an example, the goods for realizing the presentation effect of the target image may include cosmetics, makeup tools, and the like, and the service related to the target image may include makeup service and the like; for example, the electronic device may display a commodity for realizing the recommended makeup, a sales shop of the commodity, and/or a shop providing a makeup service for the recommended makeup on a display screen; for another example, the electronic device may push information of a commodity for realizing the recommended makeup, a selling shop of the commodity, and/or a shop providing a makeup service for the recommended makeup to the user device of the user, such as a purchase link of the commodity for realizing the recommended makeup, an visiting link of the selling shop of the commodity, address information or visiting link of a shop providing a makeup service for the recommended makeup, and the like.

In one embodiment, after interacting the target image with the user, the electronic device may determine preference information of the user for the target image according to the behavior implemented by the user. Take a makeup image as an example: for example, when the user selects to print the user image on which the recommended makeup is superimposed or to share the recommended makeup, it may be considered that the user's liking degree for the recommended makeup is high; for another example, when the user directly switches to another makeup without performing operations such as printing or sharing, the user may be considered to have a low preference for the recommended makeup. The electronic device may upload the preference information to a cloud end to push, by the cloud end, recommended content related to the preference information to the user, such as a purchase link of a commodity for achieving a presentation effect of the target image, an access link of a sales shop of the commodity, address information or an access link of a shop providing a service related to the target image, and the like, which are pushed by the cloud end to the user.

In one embodiment, the electronic device may perform a permission verification on the user; for example, a user may manually input an account on a touch screen of the electronic device, or scan a two-dimensional code shown on the electronic device through the user device, or implement account login on the electronic device through any other manner, and then perform authority verification on the logged-in account by the electronic device. When the user has the preset authority, the electronic device may allow the target image to be interacted with the user through the technical solution of the present specification. Wherein, the user can obtain the authority in various ways; for example, a user may purchase or redeem points for an associated membership identity to obtain the rights; for another example, the user may pay attention to a public information publishing account on the social platform of a related commodity, a shop, and the like to obtain the right; for another example, the user may obtain the rights through other manners, which is not limited in this specification.

For ease of understanding, the makeup recommendation scheme of the present specification is described in detail below in conjunction with a scenario in which makeup recommendation is made to a user based on an intelligent mirror device. Fig. 3 is a schematic diagram of a scene for recommending makeup to a user according to an exemplary embodiment. As shown in fig. 3, the smart mirror device 30 may be installed at any position such as an elevator entrance, in front of a store, beside a shelf, etc., so that the user 40 only needs to stand in front of the smart mirror device 30, and the smart mirror device 30 can recommend a suitable makeup to the user 40.

The smart mirror device 30 may be provided with a camera 31, and the camera 31 may capture an image of the user 40 and display a corresponding user image 41 on the screen 32 of the smart mirror device 30. The image acquisition of the user 40 by the camera 31 may be non-real-time, so that the content of the user image 41 is kept in the state of the acquired time and does not change with the action of the user 40; alternatively, the image capture of the user 40 by the camera 31 may be real-time or near real-time, so that the content of the user image 41 may change with the motion of the user 40, and thus the experience of the user 40 is equivalent to looking into a mirror. The following description will take the example of the camera 31 acquiring the user image 41 in real time.

Wherein, camera 31 can be the degree of depth camera for intelligent mirror surface equipment 30 can carry out background blurring or background cutout etc. to the image data who gathers, avoids other users in the background, building etc. to cause the interference. The depth camera can also more accurately identify the user characteristics such as the position, the size and the like of the five sense organs of the user 40, so that when the recommended makeup is subsequently superposed on the user image 41, the image of the recommended makeup can be more attached to the face image in the user image 41, and a better virtual makeup trying effect is realized. Of course, the camera 31 may also be a common RGB camera or other types of cameras, and this specification does not limit this.

Furthermore, the smart mirror device 30 may acquire the user image 41 in other ways besides through the camera 31. For example, the smart mirror device 30 may be built in with a bluetooth module, a WIFI module, or another type of near field communication module, so that the smart mirror device 30 may establish a near field communication connection with a user device such as a mobile phone used by the user 40, and receive the user image 41 transmitted by the user device. For another example, the smart mirror device 30 may have a network module built therein, so that the smart mirror device 30 can be connected to the cloud and obtain the user image 41 uploaded to the cloud by the user device; alternatively, the network module may connect the smart mirror device 30 to the internet and obtain the user image 41 from a cloud storage space or a social network site, etc. according to the account number of the user 40 logged in to the smart mirror device 30.

FIG. 4 is a schematic diagram of guiding a user for voice interaction according to an exemplary embodiment. As shown in fig. 4, the smart mirror device 30 may present the guidance information 42 to the user 40, for example, the guidance information 42 may include "please say your needs" to guide the user 40 to say the user voice capable of expressing his needs. In one embodiment, the guidance information 42 may also include one or more pieces of alternative requirement information, such as "what make up was popular recently", "what make up was suitable in a meeting", "what make up was suitable for a appointment", etc., as shown in fig. 4, to avoid the user 40 not knowing how to express his or her requirements when first using it. Of course, there may be multiple characterization dimensions for the user's needs, and this description is not intended to be limiting. For example, suitable occasions (such as lover appointments, friend parties, company meetings, travel out and the like), personal preferences (such as shade degree, style preference and the like) and the like can be used for representing the user requirements.

In an embodiment, the user 40 may repeat the above alternative requirement information, or may self-organize other expression forms, which is not limited in this specification. When the requirement of the user 40 is the same as or similar to the alternative requirement information, in addition to reviewing the alternative requirement information, the user 40 may locate the alternative requirement information in other manners, for example, when the user 40 says "third", the user may be used to select a third piece of alternative requirement information shown in fig. 4, that is, "what makeup is suitable for an appointment".

In an embodiment, the smart mirror device 30 may be embedded with a microphone module 33 as shown in fig. 3, and perform voice collection on the user 40 based on the microphone module 33 to obtain the user voice of the user 40. Then, the voice of the user is quickly analyzed and recognized by a voice processing chip (such as a voice AI chip) built in the smart mirror device 30, so as to determine the requirement information of the user 40, and further determine the makeup that meets the requirement information, so as to serve as the recommended makeup for the user 40.

In an embodiment, the microphone module 33 mounted on the smart mirror device 30 may adopt a microphone or a microphone array structure; when the microphone array is adopted, the microphone array structure can accurately distinguish the user voice from the user 40, and shield the voice emitted by other users or environmental noise, and the like, which is beneficial to improving the acquisition accuracy of the user voice.

In an embodiment, the smart mirror device 30 may determine the makeup according to the user's needs through a built-in processing chip (e.g., an AI chip). For example, the processing chip may obtain a makeup recommendation model obtained by pre-training, where the makeup recommendation model is obtained by training a sample user requirement and a matched sample makeup, so that after the user requirement is input into the makeup recommendation model, the makeup recommendation model may output a corresponding recommended makeup. For another example, the mapping relationship between each makeup and each user requirement may be calibrated in advance, so as to determine the recommended makeup that meets the user requirement based on the mapping relationship. Of course, the recommended makeup that meets the needs of the user can be determined in other ways, which is not limited in this specification.

Fig. 5 is a schematic diagram of an intelligent mirror device locally recommending makeup to a user according to an exemplary embodiment. As shown in fig. 5, it is assumed that the user's voice collected by the microphone module 33 is analyzed and recognized as "what make-up is suitable for appointment? ", the smart mirror device 30 may present the analyzed speech content 43 to the user 40, so that the user 40 can accurately know whether the speech recognition result of the smart mirror device 30 meets the actual requirement of the user.

Assuming that the smart mirror device 30 determines that the corresponding recommended makeup is "peach makeup" according to the user's needs as described above (e.g., "what makeup is suitable for appointment: and prompting information of peach blossom makeup, and displaying a makeup image 44 corresponding to the peach blossom makeup in an overlapping manner with the user image 41 for the user 40 to view. In one embodiment, smart mirror device 30 has an image processing chip built in it that can be used to perform image processing to enable overlay presentation of makeup images 44 and user images 41.

When the user image 41 is acquired by the camera 31 in real time, the makeup image 44 is displayed on the user image 41 in a superposed manner, so that the user 40 looks like looking at the user who has made up the makeup, and the user can know the condition of using the recommended makeup more accurately. The user 40 can also adjust the distance, angle and the like between the user 40 and the intelligent mirror device 30, and the user image 41 and the makeup image 44 are changed along with the adjustment, so that the user 40 can more carefully and accurately know the condition after the user uses the recommended makeup; in particular, when the camera 31 is a depth camera, the overlay-displayed makeup image 44 can be made to more closely fit the user image 41 based on the depth information, greatly increasing the realism of the virtual makeup image 44.

In one embodiment, the recommended makeup that meets the user's needs may not be unique, and all matching recommended makeup may be presented to the user 40 for selection by the user 40. Or, the user demand and other screening conditions can be used together for screening out the matched recommended makeup by superposing the other screening conditions; for example, by analyzing the user image 41, the user features included in the user image 41 can be extracted, and the matching recommended makeup can be determined based on the user requirements and the user features.

In an embodiment, the smart mirror device 30 may have a built-in processing chip or other functional components, which can be used to analyze the user image 41 to extract the user features contained in the user image 41. The user characteristics may include one or more of age, facial form, facial position, skin color, hair style, hair color, wearing arrangement, etc., which is not limited in this specification.

In an embodiment, the processing chip may compare the recommended makeup that meets the user requirement with the user characteristics, respectively, to screen out recommended makeup that matches the user characteristics, or recommended makeup that has a relatively higher degree of matching with the user characteristics. For example, the training sample of the makeup recommendation model may further include a matching relationship between the sample user characteristics and the sample makeup, so that after the user characteristics and the user requirements are input into the makeup recommendation model, the makeup recommendation model may output a recommended makeup that both meets the user requirements and matches the user characteristics. For another example, the mapping relationship between each makeup and each user requirement and the user characteristics in each characteristic dimension may be calibrated in advance, so as to determine the recommended makeup that matches the user characteristics based on the mapping relationship. Of course, the recommended makeup that meets the user's needs and matches the user's characteristics may also be determined in other ways, which is not limited in this specification.

In addition, the intelligent mirror device 30 can also select the recommended makeup solely according to the user characteristics, and display the corresponding makeup image 44 and the user image 41 in an overlapping manner. For example, a makeup recommendation model obtained by pre-training may be obtained, and the makeup recommendation model is obtained by training a sample user characteristic and a matched sample makeup, so that after the user characteristic is input into the makeup recommendation model, the makeup recommendation model may output a corresponding recommended makeup. For another example, the mapping relationship between each makeup and the user feature in each feature dimension may be calibrated in advance, so as to determine the recommended makeup that matches the user feature based on the mapping relationship. Of course, the recommended makeup that matches the user characteristics may be determined in other ways, and this description is not intended to limit this.

In addition to directly determining the recommended makeup by the smart mirror device 30 and displaying the makeup image 44 corresponding to the recommended makeup, the cloud server 60 may assist the smart mirror device 30 in completing the related processing. For example, fig. 6 is a schematic diagram illustrating an intelligent mirror device recommending makeup to a user with the assistance of a cloud server according to an exemplary embodiment. As shown in fig. 6, the smart mirror device 30 may upload the collected user image 41 and the user voice to the cloud server 60, and the cloud server 60 may process the user image and the user voice. The cloud server 60 may parse the user image 41 to extract user features; the cloud server 60 may analyze the user voice to obtain a voice recognition result, and further determine the user requirement; then, the cloud server 60 may determine, according to the user characteristics and the user requirements, recommended makeup that meets the user characteristics and the user requirements, and return the determined recommended makeup to the smart mirror device 30, so that the smart mirror device 30 displays the makeup image 44 and the user image 41 corresponding to the recommended makeup in an overlapping manner.

In an embodiment, the cloud server 60 may also directly perform overlay processing on the user image 41 and the makeup image 44, and return the processed makeup image to the smart mirror device 30, so that the smart mirror device 30 displays the makeup image, and the effect is equivalent to overlay display of the makeup image 44 and the user image 41. The smart mirror device 30 continuously uploads the collected user image 41 (or uploads the collected user image 41 when it is detected that the collected user image 41 changes) to the cloud server 60, and receives the makeup image returned by the cloud server 60, so that the makeup image displayed by the smart mirror device 30 changes when the user 40 changes in motion, position, angle, and the like.

It can be seen that, the above process involves a plurality of links such as extraction of user features, identification of user requirements, determination of recommended makeup, and overlay display of makeup image 44 and user image 41, which may be implemented only on smart mirror device 30 (for example, in the embodiment shown in fig. 5), may also be implemented with assistance between smart mirror device 30 and cloud server 60 (for example, in the embodiment shown in fig. 6), and may also be implemented with assistance between smart mirror device 30 and cloud server 60 for at least a part of the links, and the rest of the links, which is not limited in this specification.

In one embodiment, smart mirror device 30 may show an identifier 45 as shown in fig. 5-6, such that user 40 may download or share makeup images 44 in smart mirror device 30 with makeup images of user image 41 by activating identifier 45. Of course, the user 40 may also send out a corresponding control voice, so that the smart mirror device 30 can implement a corresponding downloading or sharing operation based on the control voice without triggering the identifier 45. The bluetooth connection can be established between the smart mirror device 30 and user equipment such as a mobile phone used by the user 40, so as to transmit and download the makeup image to the user equipment; or, the makeup image may be stored in the cloud disk or the intelligent router of the user 40 for subsequent operations; alternatively, the makeup image may be shared with other users, for example, the smart mirror device 30 may upload the makeup image to the cloud server 60 and be shared with the other users by the cloud server 60; alternatively, the makeup image may be posted to a social networking site as social information content of the user 40.

FIG. 7 is a schematic diagram of printing a user image after makeup according to an exemplary embodiment. As shown in fig. 7, the smart mirror device 30 may have a printing module 34 built therein, so that the user 40 may transmit the makeup image to the printing module 34 by triggering the printing identifier 46 shown by the smart mirror device 30 or making a corresponding control voice, and print the makeup image to form a corresponding makeup photo 70 by the printing module 34, and acquire and retain the makeup photo 70 by the user 40.

In one embodiment, the smart mirror device 30 may perform authorization verification on the user 40 to determine whether the user 40 can perform a photo printing operation. For example, the user 40 needs to log in its own account on the smart mirror device 30 in advance through manual input or mobile phone code scanning, and when the user 40 triggers the photo printing function, the smart mirror device 30 may initiate a print request to the cloud server 60, where the print request includes the logged-in account of the user 40; then, the cloud server 60 may determine whether the logged-in account includes an unused print ticket, and if the logged-in account exists, it is apparent that the user 40 has a photo printing authority, and the cloud server 60 may check out a print ticket and return a confirmation response to the smart mirror device 30, so that the smart mirror device 30 notifies the printing module 34 to perform a photo printing operation; if the print ticket does not exist under the logged account, it indicates that the user 40 does not have the photo printing authority, the cloud server 60 may not return an acknowledgement response or return a rejection response, and the smart mirror device 30 will not trigger to perform the photo printing operation.

In one embodiment, the user 40 may purchase or use redemption of points or the like to obtain the aforementioned printed coupons. In another embodiment, when the user 40 has membership rights (which can be obtained through registration, purchase, or point redemption) at a transaction platform (or transaction website) or the like associated with the smart mirror device 30, the transaction platform can issue a certain number of printed tickets to the account of the user 40 at one time or periodically issue a preset number of printed tickets. In another embodiment, after the user 40 completes the specified operation on the transaction platform, a corresponding number of printed coupons can be obtained, for example, the specified operation may include paying attention to public information issuing account numbers of specified brands or specified shops, purchasing goods of specified brands or specified shops or specified categories, and the like, which is not limited in this specification.

Indeed, in addition to printing photographs, other solutions mentioned in this specification, such as intelligent recommended makeup, downloading or sharing makeup images, etc. as shown in fig. 5-6, may be authorized for user 40 to allow the implementation of the relevant solution when user 40 has the relevant authority.

FIG. 8 is a schematic diagram of delivering interactive advertisements, according to an example embodiment. As shown in fig. 8, the smart mirror device 30 may be used as an advertisement delivery tool to realize an accurate advertisement delivery operation. For example, the cloud server 60 may count the crowd portraits of each region Based on LBS (Location Based Service); meanwhile, the intelligent mirror device 30 may have a built-in positioning module, so that the cloud server 60 may determine the interactive advertisement corresponding to the crowd images of the area according to the area where the intelligent mirror device 30 is located, and accurately deliver the interactive advertisement to the intelligent mirror device 30.

In an embodiment, the cloud server 60 may count data such as a favorite or a purchase record of the user 40, so that after the user 40 logs in the mobile phone 80 through its own account, the cloud server 60 may accurately deliver an interactive advertisement matching the favorite or the purchase record to the mobile phone 80. For example, after the smart mirror device 30 displays "peach blossom makeup" as shown in fig. 5 to 7 to the user 40, if the user 40 directly changes to other makeup, which indicates that the user 40 has a low interest level in the "peach blossom makeup", the cloud server 60 may avoid putting an interactive advertisement related to the "peach blossom makeup" to the mobile phone 80; if the user 40 performs downloading, sharing and/or printing operations on the makeup image corresponding to the "peach makeup", which indicates that the user 40 has a high interest level in the "peach makeup", the cloud server 60 may deliver an interactive advertisement related to the "peach makeup", such as related cosmetics, a cosmetics selling store, a beauty shop providing related makeup services, and the like, to the mobile phone 80.

In an embodiment, in the process that the user 40 uses the smart mirror device 30, if the user 40 performs operations such as downloading, sharing and/or printing on the make-up image corresponding to the "peach blossom make-up", the smart mirror device 30 may also directly display the product for implementing the "peach blossom make-up", the selling shop of the product, and/or the shop providing the make-up service for the "peach blossom make-up" to the user 40, so that the user may purchase the related product or go to the related shop to experience the make-up service.

FIG. 9 is a schematic block diagram of an apparatus provided in an exemplary embodiment. Referring to fig. 9, at the hardware level, the apparatus includes a processor 902, an internal bus 904, a network interface 906, a memory 908, and a non-volatile memory 910, but may also include hardware required for other services. The processor 902 reads the corresponding computer program from the non-volatile memory 910 into the memory 908 and runs it, forming the recommendation device on a logical level. Of course, besides software implementation, the one or more embodiments in this specification do not exclude other implementations, such as logic devices or combinations of software and hardware, and so on, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

Referring to fig. 10, in a software implementation, the recommending apparatus may include:

an image acquisition unit 1001 that acquires a user image;

a voice acquisition unit 1002 that acquires a user voice;

an image determining unit 1003 that determines a target image from the user voice;

and an overlay display unit 1004 for displaying the user image and the target image in an overlay manner.

Optionally, the image acquiring unit 1001 is specifically configured to:

acquiring an image of the user to obtain an image of the user;

alternatively, the image data provided by the user is acquired as the user image.

Optionally, the image determining unit 1003 is specifically configured to:

analyzing and recognizing the user voice locally or under the assistance of a cloud end to determine the requirement information of the user;

and determining a target image which accords with the requirement information.

Optionally, the image determining unit 1003 is specifically configured to:

displaying alternative requirements;

determining the selected alternative requirements as requirement information of the user according to the selection result of the user voice on the alternative requirements;

and determining a target image which accords with the requirement information.

Optionally, the image determining unit 1003 determines the target image meeting the requirement information by:

and determining a target image meeting the requirement information locally or under the assistance of a cloud.

Optionally, the requirement information includes at least one of the following: is suitable for occasions and personal preferences.

Optionally, the method further includes:

a feature extraction unit 1005 that extracts user feature information included in the user image;

wherein the target image further matches the user characteristic information.

Optionally, the feature extraction unit 1005 is specifically configured to:

analyzing the user image locally or with the assistance of a cloud to obtain the user characteristic information.

Optionally, the user characteristic information includes at least one of: age, face shape, position of five sense organs, skin color, hair style, hair color, and wearing.

Optionally, the target image includes at least one of:

makeup images, hairstyle images, headwear images, and wearing images.

Optionally, the method further includes:

a channel providing unit 1006, configured to provide a download channel or a sharing channel for the target image or the user image on which the target image is superimposed to the user.

Optionally, the method further includes:

an instruction sending unit 1007 sends a print instruction to a built-in print module or a peripheral related print device to print the target image or the user image on which the target image is superimposed.

Optionally, the method further includes:

an information interaction unit 1008 that interacts with the user for a commodity that realizes the presentation effect of the target image, a sales shop of the commodity, and/or a shop that provides a service related to the target image.

Optionally, the method further includes:

a preference determining unit 1009 that determines preference information of the user for the target image;

the preference uploading unit 1010 uploads the preference information to a cloud, so that recommended content related to the preference information is pushed to the user by the cloud.

Optionally, the method further includes:

an authority verification unit 1011 that performs authority verification on the user;

and an interaction control unit 1012, when the user has a preset right, allowing the user image and the target image to be displayed in an overlapping manner to the user.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present description to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of one or more embodiments herein. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

The above description is only for the purpose of illustrating the preferred embodiments of the one or more embodiments of the present disclosure, and is not intended to limit the scope of the one or more embodiments of the present disclosure, and any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the one or more embodiments of the present disclosure should be included in the scope of the one or more embodiments of the present disclosure.