阅读说明：本技术 扫地机器人的清扫方法及装置 (Sweeping method and device of sweeping robot ) 是由 檀冲 张书新 霍章义 王颖 李欢欢 李贝 于 2020-06-18 设计创作，主要内容包括：本发明公开了一种扫地机器人的清扫方法及装置,包括：从语音采集装置采集的声音信号中确定声音片段,所述声音片段对应的语义内容满足第一预设条件；确定所述声音片段对应的用户清扫需求；根据环境感知传感器采集的环境数据,构建所述用户清扫需求对应的实时地图；根据所述用户清扫需求以及实时地图,确定清扫路径以及所述清扫路径对应的清扫模式。本发明的技术方案通过对声音信息进行说话人识别,并根据不同的用户采用不同的清扫方式进行清扫,丰富了扫地机器人的清扫模式,满足不同的清扫要求,使得清扫过程更加智能。(The invention discloses a sweeping method and a sweeping device of a sweeping robot, which comprise the following steps: determining a sound fragment from a sound signal collected by a voice collecting device, wherein semantic content corresponding to the sound fragment meets a first preset condition; determining a user cleaning requirement corresponding to the sound clip; constructing a real-time map corresponding to the user cleaning requirement according to environmental data acquired by an environmental perception sensor; and determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the cleaning requirement of the user and the real-time map. According to the technical scheme, the speaker recognition is carried out on the voice information, and different cleaning modes are adopted for cleaning according to different users, so that the cleaning mode of the sweeping robot is enriched, different cleaning requirements are met, and the cleaning process is more intelligent.)

1. A sweeping method of a sweeping robot is characterized by comprising the following steps:

determining a sound fragment from a sound signal collected by a voice collecting device, wherein semantic content corresponding to the sound fragment meets a first preset condition;

determining a user cleaning requirement corresponding to the sound clip;

constructing a real-time map corresponding to the user cleaning requirement according to environmental data acquired by an environmental perception sensor;

and determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the cleaning requirement of the user and the real-time map.

2. The method according to claim 1, wherein the determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning requirement and a real-time map comprises:

dividing the ground area in the real-time map according to description information of a second target object in the real-time map to obtain a first cleaning area and a second cleaning area, wherein the first cleaning area indicates a ground range influenced by the second target object corresponding to the first cleaning area, and the second cleaning area indicates a ground range outside the first cleaning area;

determining a first cleaning path and a first cleaning mode corresponding to the first cleaning area according to the description information of a second target object corresponding to the first cleaning area and the cleaning requirement of the user;

and determining a second cleaning path and a second cleaning mode of the second cleaning area according to the cleaning requirement of the user.

3. The method according to claim 1, wherein the constructing a real-time map corresponding to the user sweeping requirement comprises:

acquiring an initial three-dimensional map and environmental data acquired by an environmental perception sensor;

determining a three-dimensional image corresponding to the environmental data according to the environmental data acquired by the environmental perception sensor;

and constructing a real-time map corresponding to the user cleaning requirement according to the three-dimensional image and the initial three-dimensional map.

4. The method of claim 3, wherein the constructing a real-time map corresponding to the user cleaning requirement from the three-dimensional image and the initial three-dimensional map comprises:

when the user cleaning requirement meets a second preset condition, carrying out target object identification on the three-dimensional image, and determining position information and semantic information of a first target object in the three-dimensional image;

and fusing the three-dimensional map and the initial three-dimensional map according to the position information and the semantic information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

5. The method of claim 3, wherein the constructing a real-time map corresponding to the user cleaning requirement from the three-dimensional image and the initial three-dimensional map comprises:

when the user cleaning requirement meets a third preset condition, carrying out target object identification on the three-dimensional image, and determining position information of a first target object in the three-dimensional image;

and fusing the three-dimensional map and the initial three-dimensional map according to the position information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

6. The method of claim 5, wherein the cleaning patterns corresponding to the cleaning paths are the same.

7. The method of claim 2, wherein the initial three-dimensional map comprises an empty room three-dimensional map or a historical three-dimensional map.

8. The method according to claim 1, wherein the determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning requirement and a real-time map comprises:

determining a reference cleaning mode according to the cleaning requirement of the user;

when the reference cleaning mode meets a fourth preset condition, acquiring a ground image corresponding to a ground area in the real-time map;

determining the corresponding cleanness degree of the ground area in the ground image according to the result of identifying the ground image;

and determining a cleaning path and a cleaning mode corresponding to the ground area according to the cleaning degree corresponding to the ground area.

9. The method according to any one of claims 1 to 8, wherein the environmental perception sensor comprises a vision sensor and/or a depth sensor;

the sweeping mode includes sweeping intensity and/or sweeping speed.

10. The utility model provides a cleaning device of robot sweeps floor which characterized in that includes:

the voice acquisition device comprises a fragment acquisition module, a voice acquisition module and a voice recognition module, wherein the fragment acquisition module is used for determining a voice fragment from a voice signal acquired by the voice acquisition device, and semantic content corresponding to the voice fragment meets a first preset condition;

the requirement determining module is used for determining a user cleaning requirement corresponding to the sound clip;

the map building module is used for building a real-time map corresponding to the user cleaning requirement according to the environmental data acquired by the environmental perception sensor;

and the cleaning module is used for determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning requirement and the real-time map.

Technical Field

The invention relates to the technical field of household appliances, in particular to a sweeping method and a sweeping device of a sweeping robot.

Background

Along with the development of economy and the progress of science and technology, people's requirement to living environment quality is higher and higher, and along with the rise of artificial intelligence, the market also more and more focuses on the technique of cleaning based on artificial intelligence technique, and in current intelligence cleans the technique, the robot of sweeping the floor improves people's work efficiency at home owing to can reduce people's intensity of labour at home, consequently receives extensive welcome. The floor sweeping robot is also called an automatic cleaner, intelligent dust collection, a robot dust collector and the like, is one of intelligent household appliances, and can automatically finish floor cleaning work in a room by means of certain artificial intelligence.

In the traditional automatic cleaning process, the cleaning robot usually performs cleaning operation according to a preset cleaning path in a current scene according to a preset program, and only can clean at will, and the cleaning is not intelligent enough.

Disclosure of Invention

The sweeping robot cleaning method and device provided by the invention have the advantages that speaker recognition is carried out on voice information, different cleaning modes are adopted for cleaning according to different users, the cleaning mode of the sweeping robot is enriched, different cleaning requirements are met, and the cleaning process is more intelligent.

In a first aspect, the present invention provides a cleaning method of a cleaning robot, including:

determining a sound fragment from a sound signal collected by a voice collecting device, wherein semantic content corresponding to the sound fragment meets a first preset condition;

determining a user cleaning requirement corresponding to the sound clip;

constructing a real-time map corresponding to the user cleaning requirement according to environmental data acquired by an environmental perception sensor;

and determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the cleaning requirement of the user and the real-time map.

In a second aspect, the present invention provides a cleaning device of a cleaning robot, including:

the voice acquisition device comprises a fragment acquisition module, a voice acquisition module and a voice recognition module, wherein the fragment acquisition module is used for determining a voice fragment from a voice signal acquired by the voice acquisition device, and semantic content corresponding to the voice fragment meets a first preset condition;

the requirement determining module is used for determining a user cleaning requirement corresponding to the sound clip;

the map building module is used for building a real-time map corresponding to the user cleaning requirement according to the environmental data acquired by the environmental perception sensor;

and the cleaning module is used for determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning requirement and the real-time map.

In a third aspect, the present invention provides a sweeping robot, including: the cleaning device of the sweeping robot in the second aspect, and the voice collecting device and the environmental perception sensor connected with the cleaning device of the sweeping robot.

In a fourth aspect, the invention provides a readable medium comprising executable instructions which, when executed by a processor of an electronic device, cause the electronic device to perform the method according to any one of the first aspect.

In a fifth aspect, the present invention provides an electronic device, comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method according to any one of the first aspect.

The invention provides a sweeping robot cleaning method and a device, the method determines a sound segment from a sound signal collected by a voice collecting device, semantic content corresponding to the sound segment meets a first preset condition, thereby facilitating subsequent data processing, then determines a user cleaning requirement corresponding to the sound segment, then constructs a real-time map corresponding to the user cleaning requirement according to environment data collected by an environment perception sensor, then determines a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning requirement and the real-time map, the obtained cleaning path and the cleaning mode are determined based on the real-time map and the user cleaning requirement, the method has relatively high reference value, and subsequently, when the sweeping robot cleans according to the cleaning mode and the cleaning path, the cleaning effect can be ensured, and the user cleaning requirement can be met, the user experience is improved. In conclusion, according to the technical scheme provided by the invention, the speaker recognition is carried out on the voice information, and different cleaning modes are adopted for cleaning according to different users, so that the cleaning mode of the sweeping robot is enriched, different cleaning requirements are met, and the cleaning process is more intelligent.

Further effects of the above-mentioned unconventional preferred modes will be described below in conjunction with specific embodiments.

Drawings

In order to more clearly illustrate the embodiments or the prior art solutions of the present invention, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.

Fig. 1 is a schematic flow chart of a cleaning method of a cleaning robot according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of another cleaning method of a cleaning robot according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a cleaning device of a sweeping robot according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a sweeping robot according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail and completely with reference to the following embodiments and accompanying drawings. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Fig. 1 shows a specific embodiment of a cleaning method of a sweeping robot according to the present invention. In this embodiment, the method specifically includes the following steps:

step 101, determining a sound segment from a sound signal collected by a voice collection device, wherein semantic content corresponding to the sound segment meets a first preset condition.

In particular, the speech acquisition means particularly refers to a device capable of acquiring sound, such as a microphone array. The voice acquisition device can be installed on the sweeping robot or in a room, when the voice acquisition device is installed in the room, the voice signal acquired by the voice acquisition device needs to be sent to the sweeping robot, and therefore the sweeping robot can timely know the speaking content of a user.

Specifically, the sound signal refers to an analog signal of the amplitude of the sound wave of the external environment, which is collected by the voice collecting device, changing with time; the sound segment specifically refers to a signal segment in which semantic content satisfying a first preset condition corresponds to a sound signal. It should be noted that there may be a plurality of sound segments, and semantic content corresponding to each sound segment satisfies the first preset condition.

Specifically, the semantic content meeting the first preset condition specifically means that the semantic content includes a preset cleaning control instruction, the preset cleaning instruction may include one or more, and the specific requirement is determined by combining an actual requirement, for example, the preset cleaning control instruction may be a word such as cleaning or sweeping. Optionally, the semantic content corresponding to the sound clip includes a preset cleaning control instruction, so as to ensure that the sound clip corresponds to a user, thereby facilitating data processing.

Specifically, a voice signal acquired by a voice acquisition device is acquired, the voice signal is subjected to semantic recognition to obtain semantic content of the voice signal, the semantic content of the voice signal is matched with a preset cleaning control instruction, whether the received voice signal contains the preset cleaning control instruction is judged, if the received voice signal contains the preset cleaning control instruction is judged, the semantic content of the voice signal contains the preset cleaning control instruction, a segment of the voice signal corresponding to the semantic content corresponding to the preset cleaning control instruction is determined, and the segment is determined to be a voice segment.

And 102, determining a user cleaning requirement corresponding to the sound clip.

Specifically, the user cleaning requirement may indicate the final effect of cleaning the room, for example, if the user cleaning requirement is high, it indicates that the cleanliness of the room should be high after cleaning the room.

Specifically, for each sound clip, voiceprint information in the sound clip is extracted, voiceprint information and a plurality of preset candidate voiceprint information are matched, and therefore the user corresponding to the sound clip is judged as which one of prestored users corresponding to the plurality of preset candidate voiceprint information respectively, the user corresponding to the sound clip is determined, the prestored user corresponding to the each preset candidate voiceprint information is provided with a user cleaning requirement, and based on the corresponding relation between the prestored user corresponding to the plurality of preset candidate voiceprint information and the cleaning requirements of the plurality of users, the user cleaning requirement corresponding to the sound clip is determined. It should be noted that, if the user corresponding to the voice clip is not a pre-stored user corresponding to each of the plurality of preset candidate voiceprint information, the cleaning robot does not perform the cleaning operation.

It should be further noted that, when there are a plurality of sound segments, each sound segment corresponds to one user, and these users may be the same, may also be different, or may also be partially the same, and the user's cleaning requirement may be selected to have the highest cleaning requirement, so as to ensure that the cleaning requirements of all users are met, and the user experience is improved.

Alternatively, the user corresponding to the sound segment may be determined by a voiceprint recognition model. The voiceprint recognition model can be obtained through training of the training set, the training set comprises a plurality of voice signals of pre-stored users in different states, the voice signals of the non-pre-stored users can be included, the accuracy of the users identified by the voiceprint recognition model is ensured, the sweeping robot can collect the voice signals at any time, the voice signals of the pre-stored users of the plurality of the training set are continuously updated, the voiceprint recognition model is updated by the aid of the updated training set, the voiceprint recognition model is continuously optimized, when the pre-stored users are newly added, the voice signals of the pre-stored users can be increased at any time in the training set, voiceprint features capable of being recognized are enlarged, and the sweeping robot is enabled to have self-learning ability.

And 103, constructing a real-time map corresponding to the cleaning requirement of the user according to the environmental data acquired by the environmental perception sensor.

In the embodiment of the invention, the sweeping robot is provided with the environment sensing sensor, the environment sensing sensor is used for collecting environment data, the real-time map corresponding to the sweeping requirement of the user is constructed based on the environment data, the real-time map takes the current environment condition into consideration, the reference value is higher, meanwhile, different user sweeping requirements can correspond to different real-time maps, and the sweeping path planning methods respectively corresponding to different real-time maps are different, so that the real-time map is adjusted according to the user sweeping requirements to adapt to different scenes.

Specifically, the environment sensing sensor is configured to sense environment information around the sweeping robot, optionally, the environment sensing sensor includes a visual sensor and/or a depth sensor, and correspondingly, the environment data includes a depth image acquired by the depth sensor and/or an indoor image acquired by the visual sensor; the visual sensor is specifically an apparatus for acquiring image information of an external environment by using an optical element and an imaging device, and may be an RGB camera, which may be a monocular camera or a fisheye camera, and is not limited herein, and any RGB camera in the prior art may be used. The depth sensor specifically refers to an instrument for collecting a distance from an outer ring environment to the depth sensor, and may be a structured light sensor, a laser sensor, or the like. The structured light sensor refers to any instrument capable of projecting structured light and collecting the projected structured light, and optionally, the structured light sensor includes a structured light emitter for projecting structured light outwards and a structured light receiver for collecting structured light projected by the structured light emitter reflected by a subject. In particular, the structured light emitter comprises a line structured light emitter or a smooth structured light emitter, preferably a line structured light emitter. The structured light receiver is preferably a monocular infrared camera.

Optionally, the real-time map of the embodiment of the present invention may be determined by the following method:

acquiring an initial three-dimensional map; determining a three-dimensional image corresponding to the environmental data according to the environmental data acquired by the environmental perception sensor; and constructing a real-time map corresponding to the cleaning requirement of the user according to the three-dimensional image and the initial three-dimensional map.

Specifically, the initial three-dimensional map may be an empty room three-dimensional map or a historical three-dimensional map, which is not specifically limited herein and specifically needs to be determined in combination with actual needs. The three-dimensional map of the empty room refers to a three-dimensional map of a room where the sweeping robot is currently located or all rooms of a house where a user lives when no furniture or electric appliances exist, and preferably the three-dimensional map of all rooms of the house where the user lives, so that the initial three-dimensional map is called once, and when the initial three-dimensional map is the three-dimensional map of the empty room, the real-time map can be understood as a map obtained by fusing a three-dimensional image into the three-dimensional map of the empty room; the historical three-dimensional map refers to a previously stored three-dimensional map of a room where the sweeping robot is currently located or all rooms of a house where the user lives, and comprises furniture and electric appliances, and the three-dimensional map of all rooms of the house where the user lives is preferably selected, so that the initial three-dimensional map only needs to be called once, and when the initial three-dimensional map is the historical three-dimensional map, the real-time map can be understood as a map obtained by updating the historical three-dimensional map through the three-dimensional image. It should be noted that, the three-dimensional map of the empty room does not contain furniture and electric appliances, so that the occupied storage capacity is small, the storage capacity of the sweeping robot can be saved, the calculation amount is reduced, and meanwhile, the three-dimensional map of the empty room can be flexibly arranged to facilitate data processing, so that the three-dimensional map of the empty room is preferred.

Specifically, the three-dimensional image is obtained by three-dimensionally reconstructing the environmental data. The three-dimensional reconstruction can adopt an active three-dimensional reconstruction method including any one or more of a moire fringe method, a structured light method, a triangular distance measurement method and a time-of-flight method, and can also adopt a passive three-dimensional reconstruction method including any one or more of a texture restoration shape method, a shadow restoration shape method and a stereoscopic vision method, and the specific requirement is determined based on the type of the environment perception sensor. Three-dimensional reconstruction is the prior art, and this principle is not described in detail in the embodiment of the present invention.

When the real-time map is a map with semantics, optionally, carrying out target object identification on the three-dimensional image, and determining position information and semantic information of a first target object in the three-dimensional image; and fusing the three-dimensional map and the initial three-dimensional map according to the position information and the semantic information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

Specifically, in order to ensure that the three-dimensional image can truly reflect the actual situation of the room, the first object in the three-dimensional image specifically refers to all objects in the three-dimensional image, such as furniture, appliances, floors, walls, and the like. The position information of the first object specifically refers to three-dimensional coordinates of the first object in the three-dimensional image, and may include three-dimensional coordinates of a plurality of reference points constituting the first object, so as to better describe the size and shape of the object. Specifically, the semantic information of the first object specifically refers to the object type.

When the real-time map is a map without semantics, optionally, performing target object identification on the three-dimensional image, and determining position information of a first target object in the three-dimensional image; and fusing the three-dimensional map and the initial three-dimensional map according to the position information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

Specifically, the three-dimensional image is identified by an image identification method, wherein the image identification method can adopt a three-dimensional image identification method in the prior art, and the implementation of the invention does not need to give redundant details to the principle of three-dimensional image identification.

Specifically, the fact that the user cleaning requirement meets the second preset condition specifically means that the user cleaning requirement is relatively high; the fact that the user cleaning requirement meets the third preset condition specifically means that the user cleaning requirement is relatively low. For example, when the user's cleaning requirement is relatively low, a real-time map without semantics may be selected, and when the user's cleaning requirement is high, a real-time map with semantics may be selected. Wherein, the real-time map is preferably a three-dimensional map.

And step 104, determining a cleaning path and a cleaning mode corresponding to the cleaning path according to the cleaning requirement of the user and the real-time map.

Specifically, the cleaning path specifically refers to a movement track of the sweeping robot during cleaning operation. The sweeping mode is preset, and the sweeping mode may include several modes, such as a fast sweeping mode, a deep sweeping mode, a soft sweeping mode, a power saving sweeping mode, and a standard sweeping mode. Each sweep mode includes, but is not limited to, sweep speed and sweep intensity. The cleaning speed is a speed parameter during cleaning, and the slower the cleaning speed is, the cleaner the cleaning is, the slower the cleaning speed corresponding to the deep cleaning mode is, and the faster the cleaning speed corresponding to the fast cleaning mode is. The cleaning intensity refers to the acting force of the cleaning structure on the ground during cleaning, the cleaning intensity is larger, the cleaning intensity corresponding to a deep cleaning mode is larger, the cleaning intensity corresponding to a quick cleaning mode is smaller, and the cleaning structure comprises an edge brush, a rolling brush, a sundries box, a motor and the like which are similar to a dust collector.

As a possible case, when the real-time map has no semantics, a cleaning mode according to which cleaning of a room is completed may be determined according to a user cleaning demand.

In order to ensure the cleaning effect, optionally, cleaning may be specifically performed by:

determining a reference cleaning mode according to the cleaning requirement of a user; when the reference cleaning mode meets a fourth preset condition, acquiring a ground image corresponding to a ground area in the real-time map; determining the corresponding cleanliness of the ground area in the ground image according to the result of identifying the ground image; and determining a cleaning path and a cleaning mode corresponding to the ground area according to the cleaning degree corresponding to the ground area.

Specifically, it specifically means to refer to the mode of cleaning and satisfy the fourth preset condition, refer to the mode of cleaning and belong to appointed mode of cleaning, in order to ensure to clean the effect, appointed mode of cleaning can be the relatively poor mode of cleaning the effect, for example, clean the mode fast or the mode of cleaning softly, it is usually different to consider the condition on ground, it is unnecessary to clean vigorously to some ground, appointed mode of cleaning also can be the mode that the effect is very good to clean, for example, powerful mode of cleaning, thereby ensure to clean the effect to the dirty region in ground, also can carry out simple quick cleaning to comparatively clean ground simultaneously, thereby promote and clean efficiency, satisfy the user and clean the demand, and user experience is improved.

Specifically, the ground image is a three-dimensional image. The ground image is identified by a three-dimensional image identification method, wherein the three-dimensional image identification method can adopt a method in the prior art, and the implementation of the invention does not need to be repeated for much.

Specifically, by presetting a mapping relationship between the cleaning degree and the cleaning mode, a preset relationship between the cleaning degree and the cleaning mode corresponding to each pre-stored user is obtained, the mapping relationship indicates the cleaning mode corresponding to different cleaning degrees, the preset relationship corresponding to each pre-stored user may be the same or different, the specific division of the cleaning degree and the type of the cleaning mode need to be determined in combination with an actual scene, and the cleaning mode that can be realized by the cleaning robot, for example, the cleaning degree may include a1, a2, A3, a4, and a5, the cleaning mode corresponding to the user a in turn may be V1, V2, V3, V4, and V5, the cleaning mode corresponding to the user B in turn may be no cleaning, V1, V2, V3, and V4, where V1 to V5 represent different cleaning modes. In particular, the degree of cleanliness may be designated by a number or a letter, thereby facilitating data processing.

It should be noted that there may be a plurality of ground areas, the corresponding clean degree of each ground area is similar, and the ground areas are formed by a closed line, where all the grounds in the real-time map need to be clustered, and the grounds with similar clean degrees are gathered together to form the ground areas, so that the ground conditions can be accurately known, and thus the cleaning path of the ground area can be more accurately planned and the cleaning mode can be determined, and here, the clean degree of the ground is detected in real time, and the cleaning path and the cleaning mode are determined based on the clean degree, so that when the floor sweeping robot cleans the ground areas corresponding to the cleaning path and the cleaning mode, the cleaning effect can be ensured, and the user experience is improved.

In the embodiment of the invention, under the condition that the cleaning effect of the cleaning mode is not ideal, the cleaning effect of the ground can be ensured and the user experience can be improved by detecting the cleaning degree of the ground in real time and determining the cleaning path and the cleaning mode according to the cleaning degree. On the premise that the cleaning effect of the cleaning mode is ideal, the cleaning degree of the ground can be detected in real time, and the cleaning path and the cleaning mode can be determined according to the cleaning degree, so that the cleaning efficiency of the ground is ensured, and the user experience is improved.

In order to ensure the cleaning effect, optionally, cleaning may be specifically performed by:

dividing the ground area in the real-time map according to the description information of the second target object in the real-time map to obtain a first cleaning area and a second cleaning area, wherein the first cleaning area indicates the ground range influenced by the corresponding second target object, and the second cleaning area indicates the ground range outside the first cleaning area; determining a first cleaning path and a first cleaning mode corresponding to the first cleaning area according to the description information of a second target object corresponding to the first cleaning area and the cleaning requirement of a user; and determining a second cleaning path and a second cleaning mode of the second cleaning area according to the cleaning requirement of the user.

In particular, the description information specifically refers to relevant information describing the second object, which may be, for example, a position, a type, a color, a texture, a fixed object, or a movable object.

It should be noted that, considering that the number of objects in the real-time map may be many, in order to reduce the calculation amount of the object with a small reference value or no reference value, the second object specifically refers to a designated object, so as to reduce the unnecessary calculation amount, and the ground area in the real-time map may be divided more quickly. When the environment perception sensor only comprises the visual sensor, at the moment, the ground image is directly acquired by the visual sensor.

When the real-time map has semantic information, the description information of the second object may include semantic information and position information, and the position information includes, but is not limited to, three-dimensional coordinates of contour points of the second object, in this case, the first cleaning area may be determined based on the type and contour position of the second object, and specifically indicates a ground area affected by the second object, and the influence of different second objects on the cleanness of the ground area is different, optionally, an expansion factor of an area formed by the contour of the second object is set, so as to obtain first cleaning areas corresponding to different second objects, where when expanding according to the expansion factor, the middle point of the area formed by the contour of the second object should be used as a base point for expanding, for example, the ground area that can be affected by the dining table is relatively large, in this case, a relatively large expansion factor may be set, at this time, the area formed by the outline of the dining table is the table top of the dining table, the area formed after expansion is the ground area influenced by the dining table by taking the central point of the table top as a base point, the ground area influenced by the wardrobe is relatively small, at this time, a relatively small expansion multiple can be set, at this time, the area formed by the outline of the wardrobe is the ground of the wardrobe, the area formed after expansion is the ground area influenced by the wardrobe by taking the central point of the ground as the base point according to the set expansion multiple. The first cleaning modes respectively corresponding to each first cleaning area can be the same or different, the description information of the second target object and the cleaning requirement of the user are specifically required to be set in advance, and the first cleaning path and the first cleaning mode are matched with each other, so that the cleaning requirement of the user is met, and an ideal cleaning effect is achieved.

When the real-time map does not have semantic information, the description information of the second object may include position information, and the position information includes, but is not limited to, three-dimensional coordinates of contour points of the second object, and at this time, the first cleaning area may be determined based on the contour position of the second object, and specifically indicates the ground area affected by the second object, and here, since the type of the second object is not considered, the ground area affected by the second object may be considered to be the same, and optionally, an expansion multiple of an area formed by the contour of the second object is set, so as to obtain the first cleaning area corresponding to the second object, and the expansion multiple of each second object is the same. It should be noted that the first cleaning modes corresponding to each first cleaning region are the same, and the first cleaning mode and the second cleaning mode may be the same or different, and specifically need to be set in advance in combination with an actual scene.

It should be noted that the first cleaning area and the second cleaning area should be determined according to the self-attribute of the second target object, and do not depend on the cleaning requirement of the user; the planning of the first cleaning path mainly depends on the self attribute of the second target object, and the size of the cleaning area, the distance between adjacent paths and the overlapping area can be properly adjusted according to the cleaning requirement of a user; the cleaning mode mainly depends on the cleaning requirement of the user, and the cleaning intensity or the cleaning speed can be properly adjusted according to the condition of the first cleaning path.

It should be noted that, when planning the second cleaning area, the starting point or the ending point of the first cleaning path should be taken as a reference, so as to ensure that the second cleaning path and the first cleaning path are continuous, and ensure that the sweeping robot can clean the first cleaning area and the second cleaning area orderly.

According to the technical scheme, the beneficial effects of the embodiment are as follows: the speaker recognition is carried out on the voice signals of the users, so that the cleaning requirements of different users are determined, and the sweeping robot can process the cleaning requirements of different users; the method comprises the steps of reconstructing a three-dimensional map through an environment perception sensor, obtaining a real-time map corresponding to a user, and constructing different real-time maps based on the cleaning requirements of different users; adopt different modes of cleaning through different users, and adopt different real-time map plans to clean the route to realize that different users adopt different modes of cleaning to clean, ensured to clean the effect, satisfy different requirements of cleaning, richened the mode of cleaning of robot of sweeping the floor, make the process of cleaning more intelligent.

Fig. 1 shows only a basic embodiment of the method of the present invention, and based on this, certain optimization and expansion can be performed, and other preferred embodiments of the method can also be obtained.

To more clearly illustrate the technical solution of the present invention, referring to fig. 2, an embodiment of the present invention provides another cleaning method for a cleaning robot, and the present embodiment is further described with reference to specific application scenarios on the basis of the foregoing embodiment, where the specific scenarios are as follows: the high place is said to be "please clean the room", the high place is a family member, and the house in which the high place is located has 3 rooms, which are respectively A1, A2 and A3. In this embodiment, the method may specifically include the following steps:

step 201, determining a sound segment from a sound signal collected by a voice collection device, wherein semantic content corresponding to the sound segment meets a first preset condition.

Supposing that the voice acquisition device is arranged on the floor sweeping robot, the voice acquisition device acquires a voice signal generated by a high certain speech, and carries out semantic recognition on the voice signal, wherein the obtained semantic content corresponding to the voice signal is as follows: please clean the room, and at the same time, judge that the semantic content corresponding to the sound signal includes a preset cleaning command of "cleaning", at this time, determine the segment corresponding to the sound signal of "cleaning" as the sound segment.

Step 202, determining a user cleaning requirement corresponding to the sound clip.

Specifically, the voice segment is input into a voiceprint recognition model, and the voiceprint recognition model can extract voiceprint information in the voice segment and recognize a user corresponding to the voiceprint information, so that the user corresponding to the voice segment is higher; or extracting voiceprint information corresponding to the voice fragment, matching the voiceprint information with a pre-constructed voiceprint database, and determining that the user corresponding to the voice fragment is higher. The cleaning requirement set by each user in advance, and therefore, the cleaning requirement corresponding to a higher one is determined as the cleaning requirement of the user.

Step 203, acquiring an initial three-dimensional map and environmental data acquired by an environmental perception sensor; and determining a three-dimensional image corresponding to the environmental data according to the environmental data acquired by the environmental perception sensor.

Assuming that the sweeping robot is provided with the RGB camera and the structured light sensor, at this time, a three-dimensional map of empty rooms from a room a1 to A3 can be obtained, then an indoor image acquired by the RGB camera and an indoor depth image acquired by the structured light sensor are obtained, where the indoor image and the indoor depth image are corresponding to each other, in other words, the indoor depth image includes depth information corresponding to the indoor image, and then three-dimensional reconstruction is performed based on the depth image and the indoor image to obtain a three-dimensional image.

Step 204, when the cleaning requirement of the user meets a second preset condition, carrying out target object identification on the three-dimensional image, and determining position information and semantic information of a first target object in the three-dimensional image; and fusing the three-dimensional map and the initial three-dimensional map according to the position information and the semantic information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

And if the cleaning requirement of the user is high, image recognition can be carried out on the three-dimensional map to obtain position information and semantic information respectively corresponding to all target objects in the three-dimensional map, and based on the position information and the semantic information, the three-dimensional map is fused into the three-dimensional map of the empty room to obtain the real-time map with semantics.

Step 205, dividing the ground area in the real-time map according to the description information of the second target object in the real-time map to obtain a first cleaning area and a second cleaning area, where the first cleaning area indicates a ground area affected by the second target object corresponding to the first cleaning area, and the second cleaning area indicates a ground area outside the first cleaning area.

Presetting a plurality of second targets, acquiring semantic information, three-dimensional coordinates of contour points, expansion multiples of the three-dimensional coordinates and contour line attributes of the second targets in a real-time map for each second target, expanding the three-dimensional coordinates according to the expansion multiples of the three-dimensional coordinates to obtain expanded three-dimensional coordinates, performing curve fitting on the expanded three-dimensional coordinates according to the contour line attributes and the expanded three-dimensional coordinates to obtain a ground area influenced by the second targets, and determining the ground area as a first cleaning area. And then, determining the ground area except the first cleaning area corresponding to all the second targets in the real-time map as a second cleaning area.

Step 206, determining a first cleaning path and a first cleaning mode corresponding to the first cleaning area according to the description information of a second target object corresponding to the first cleaning area and the cleaning requirement of the user; and determining a second cleaning path and a second cleaning mode of the second cleaning area according to the cleaning requirement of the user.

Determining a first cleaning path corresponding to the first cleaning area according to the type of the second target object; then, according to the cleaning requirement of the user, a first cleaning mode is determined, and the sweeping robot cleans the first cleaning area according to the first cleaning path and the first cleaning mode.

According to the technical scheme, the beneficial effects of the embodiment are as follows: the speaker recognition is carried out on the voice signal of the user, so that the cleaning requirement of the user is determined; reconstructing a three-dimensional map through an environment perception sensor to obtain a real-time map which meets the requirements of a user and has semantics; through the description information of the target object in the real-time map, the area that can be influenced by the target object and the area that can not be influenced are determined, the cleaning mode corresponding to the cleaning requirement of the cleaning path and the user is determined according to different areas, different cleaning paths and cleaning modes are adopted for cleaning different ground surfaces, the cleaning requirements of different users are met by adopting different cleaning modes, the cleaning effect is ensured simultaneously, the cleaning mode of the sweeping robot is enriched, and the cleaning process is more intelligent.

Based on the same concept as the method embodiment of the present invention, referring to fig. 3, an embodiment of the present invention further provides a cleaning device of a sweeping robot, including:

the segment acquiring module 301 is configured to determine a sound segment from a sound signal acquired by a voice acquisition device, where semantic content corresponding to the sound segment meets a first preset condition;

a requirement determining module 302, configured to determine a user cleaning requirement corresponding to the sound clip;

the map building module 303 is configured to build a real-time map corresponding to the user cleaning requirement according to the environmental data acquired by the environmental sensing sensor;

and a cleaning module 304, configured to determine a cleaning path and a cleaning mode corresponding to the cleaning path according to the user cleaning demand and a real-time map.

In addition, on the basis of the embodiment shown in fig. 3, in the embodiment of the present invention, the cleaning module 304 includes: the cleaning device comprises a dividing unit, a first cleaning unit and a second cleaning unit; wherein the content of the first and second substances,

the dividing unit is used for dividing the ground area in the real-time map according to description information of a second target object in the real-time map to obtain a first cleaning area and a second cleaning area, wherein the first cleaning area indicates a ground area influenced by the second target object corresponding to the first cleaning area, and the second cleaning area indicates a ground area outside the first cleaning area;

the first cleaning unit is used for determining a first cleaning path and a first cleaning mode corresponding to the first cleaning area according to the description information of a second target object corresponding to the first cleaning area and the cleaning requirement of the user;

the second cleaning unit is used for determining a second cleaning path and a second cleaning mode of the second cleaning area according to the cleaning requirement of the user.

In an embodiment of the present invention, the cleaning module 304 includes: the device comprises a mode determining unit, an image acquiring unit, an image identifying unit and a third cleaning unit; wherein the content of the first and second substances,

the mode determining unit is used for determining a reference cleaning mode according to the cleaning requirement of the user;

the image acquisition unit is used for acquiring a ground image corresponding to a ground area in the real-time map when the reference cleaning mode meets a fourth preset condition;

the image identification unit is used for determining the corresponding cleanness degree of the ground area in the ground image according to the result of identifying the ground image;

and the third cleaning unit is used for determining a cleaning path and a cleaning mode corresponding to the ground area according to the cleaning degree corresponding to the ground area.

In this embodiment of the present invention, the map building module 303 includes: the map building device comprises an acquisition unit, an image determining unit and a map building unit; wherein the content of the first and second substances,

the acquisition unit is used for acquiring an initial three-dimensional map and environmental data acquired by an environmental perception sensor;

the image determining unit is used for determining a three-dimensional image corresponding to the environmental data according to the environmental data acquired by the environmental perception sensor;

and the map construction unit is used for constructing a real-time map corresponding to the user cleaning requirement according to the three-dimensional image and the initial three-dimensional map.

In an embodiment of the present invention, the map building unit includes: a first identification subunit and a first construction subunit; wherein the content of the first and second substances,

the first identification subunit is configured to, when the user cleaning requirement meets a second preset condition, perform target object identification on the three-dimensional image, and determine position information and semantic information of a first target object in the three-dimensional image;

the first construction subunit is configured to fuse the three-dimensional map and the initial three-dimensional map according to the position information and semantic information of the first target object in the three-dimensional image, so as to construct a real-time map corresponding to the user cleaning requirement.

In an embodiment of the present invention, the map building unit includes: a second identification subunit and a second construction subunit; wherein the content of the first and second substances,

the second identification subunit is configured to, when the user cleaning demand meets a third preset condition, perform target object identification on the three-dimensional image, and determine position information of a first target object in the three-dimensional image;

and the second construction subunit is used for fusing the three-dimensional map and the initial three-dimensional map according to the position information of the first target object in the three-dimensional image so as to construct a real-time map corresponding to the cleaning requirement of the user.

In an embodiment of the invention, the environmental perception sensor comprises a vision sensor and/or a depth sensor;

the sweeping mode includes sweeping intensity and/or sweeping speed.

In order to implement the above embodiment, referring to fig. 4, an embodiment of the present invention provides a sweeping robot, including: the cleaning device 403 of the sweeping robot as provided in the second embodiment of the invention, and the voice collecting device 402 and the environmental sensor 401 connected to the cleaning device of the sweeping robot.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. On the hardware level, the electronic device includes a processor 501 and a memory 502 storing execution instructions, and optionally includes an internal bus 503 and a network interface 504. The memory 502 may include a memory 5021, such as a Random-access memory (RAM), and may further include a non-volatile memory 5022(non-volatile memory), such as at least 1 disk memory; the processor 501, the network interface 504, and the memory 502 may be connected to each other by an internal bus 503, and the internal bus 503 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (extended Industry Standard Architecture) bus, or the like; the internal bus 503 may be divided into an address bus, a data bus, a control bus, etc., and is indicated by only one double-headed arrow in fig. 5 for convenience of illustration, but does not indicate only one bus or one type of bus. Of course, the electronic device may also include hardware required for other services. When the processor 501 executes execution instructions stored by the memory 502, the processor 501 performs a method in any of the embodiments of the present invention and at least is used to perform the method as shown in fig. 1 or fig. 2.

In a possible implementation manner, the processor reads the corresponding execution instruction from the nonvolatile memory to the memory and then runs the execution instruction, and the corresponding execution instruction can also be obtained from other equipment, so that the cleaning device of the cleaning robot is formed on a logic level. The processor executes the execution instruction stored in the memory, so that the cleaning method of the sweeping robot provided by any embodiment of the invention is realized through the executed execution instruction.

The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Embodiments of the present invention further provide a computer-readable storage medium, which includes an execution instruction, and when a processor of an electronic device executes the execution instruction, the processor executes a method provided in any one of the embodiments of the present invention. The electronic device may specifically be the electronic device shown in fig. 5; the execution instruction is a computer program corresponding to the cleaning device of the sweeping robot.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.

The embodiments of the present invention are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.