阅读说明：本技术 机器人控制方法、装置、机器人和存储介质 (Robot control method, device, robot and storage medium ) 是由 马帅 唐旋来 杨亚运 于 2021-08-31 设计创作，主要内容包括：本申请实施例公开了一种机器人控制方法、装置、机器人和存储介质,涉及控制领域。该方法包括：在机器人所在区域的电梯门开启后,根据预设路径,控制机器人移动且自转,以采集轿厢内部的至少两个视场角度的目标图像；依次对各所述目标图像进行人物标定和追踪,并根据标定和追踪结果,确定所述轿厢中的目标乘梯人数；根据所述目标乘梯人数,对所述机器人进行乘梯控制。本申请提高了确定结果准确度,同时避免了机器人等待时间过长对机器人,同时减少了无效乘梯过多对电梯的作业效率的影响,从而提高了机器人和电梯的作业效率。(The embodiment of the application discloses a robot control method and device, a robot and a storage medium, and relates to the field of control. The method comprises the following steps: after an elevator door of an area where the robot is located is opened, controlling the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car; sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking result; and controlling the robot to take the elevator according to the number of the target elevator taking people. The method and the device improve the accuracy of the determined result, simultaneously avoid the robot from waiting for too long, and reduce the influence of invalid elevator taking on too much operation efficiency of the elevator, thereby improving the operation efficiency of the robot and the elevator.)

1. A robot control method, performed by a robot, comprising:

after an elevator door of an area where the robot is located is opened, controlling the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car;

sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking result;

and controlling the robot to take the elevator according to the number of the target elevator taking people.

2. The method of claim 1, wherein the sequentially performing person targeting and tracking on each of the target images comprises:

processing the target image according to a preset root model and each component model to obtain target response data;

and according to the target response data, carrying out person calibration and tracking on the target image.

3. The method of claim 2, wherein the processing the target image according to the preset root model and each component model to obtain target response data comprises:

extracting the features of the target image to obtain initial feature data;

processing the initial characteristic data according to the root model and each component model respectively to obtain initial response data;

and generating the target response data according to each initial response data.

4. The method of claim 2, wherein said person targeting and tracking said target image based on said target response data comprises:

processing the target response data based on a classification model to obtain a classification result;

and calibrating the newly added people and tracking the calibrated people according to the target position information corresponding to the classification result.

5. The method of claim 1, wherein determining a target number of passengers in the car based on the calibration and tracking results comprises:

determining the current elevator taking number in the elevator car according to the calibration and tracking result;

and determining the target elevator taking number in the elevator car according to the current elevator taking number and the historical elevator taking number.

6. The method of claim 5, wherein determining a target number of passengers in the car based on the current number of passengers and a historical number of passengers comprises:

if the current elevator taking number and the historical elevator taking number tend to be stable, taking the current elevator taking number as the target elevator taking number; and if not, taking the current elevator taking number as the historical elevator taking number, and re-determining the current elevator taking number according to the re-collected target image.

7. The method of claim 5, wherein determining a target number of passengers in the car based on the target image comprises:

determining the current elevator taking number in the elevator car according to the target image;

and determining the number of the target elevator taking persons according to the statistical result of the elevator taking in and out and the current number of the elevator taking persons.

8. A robot control device, which is provided in a robot, includes:

the target image acquisition module is used for controlling the robot to move and rotate according to a preset path after an elevator door of an area where the robot is located is opened so as to acquire target images of at least two view field angles in the elevator car;

the target elevator taking number determining module is used for sequentially calibrating and tracking people of each target image and determining the target elevator taking number in the elevator car according to the calibration and tracking result;

and the elevator taking control module is used for controlling the robot to take the elevator according to the number of the target elevator taking people.

9. A robot, comprising:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement a robot control method as recited in any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a robot control method according to any one of claims 1-7.

Technical Field

The embodiment of the application relates to the field of control, in particular to a robot control method and device, a robot and a storage medium.

Background

With the continuous development of the technology, robots are distributed in various fields such as catering, medical treatment, hotels, logistics distribution and the like. In order to enlarge the moving area of the robot during the operation of the robot, the robot is usually required to take an elevator to realize cross-floor service.

In the prior art, when a robot takes an elevator, whether an elevator car is empty needs to be detected, and the robot takes the elevator to continue working under the condition that the elevator car is empty. However, the above method will result in too long waiting time of the robot and affect the working efficiency of the robot and the elevator.

Disclosure of Invention

The application provides a robot control method, a robot control device, a robot and a storage medium, so that when the robot takes an elevator to work, the working efficiency of the robot and the elevator is improved.

In a first aspect, an embodiment of the present application provides a robot control method, performed by a robot, including:

after an elevator door of an area where the robot is located is opened, controlling the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car;

sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking result; and controlling the robot to take the elevator according to the number of the target elevator taking people.

In a second aspect, an embodiment of the present application further provides a robot control device configured for a robot, including:

the target image acquisition module is used for controlling the robot to move and rotate according to a preset path after an elevator door of an area where the robot is located is opened so as to acquire target images of at least two view field angles in the elevator car;

and the target elevator taking number determining module is used for sequentially calibrating and tracking people of each target image, determining a target elevator taking number taking control module in the elevator car according to the calibration and tracking result, and controlling the robot to take the elevator according to the target elevator taking number.

In a third aspect, an embodiment of the present application further provides a robot, including:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement a robot control method as provided in embodiments of the first aspect of the application.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a robot control method as provided in the first aspect of the present application.

According to the embodiment of the application, after an elevator door of an area where the robot is located is opened, the robot is controlled to move and rotate according to a preset path so as to acquire targets of at least two view field angles in a car; sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator taking people in the elevator car according to the calibration and tracking result; and controlling the robot to take the elevator according to the number of the target elevator taking people. According to the technical scheme, the robot obtains the target image and determines the number of the elevator passengers, so that the accuracy of the determination result is improved. Meanwhile, the number of people taking the elevator based on the target replaces the mode of whether the elevator car is empty or not to carry out elevator taking control on the robot, so that the robot is prevented from being influenced by overlong waiting time, and meanwhile, the influence of too much invalid elevator taking on the operation efficiency of the elevator is reduced, and the operation efficiency of the robot and the elevator is improved.

Drawings

Fig. 1 is a flowchart of a robot control method provided in an embodiment of the present application;

fig. 2A is a flowchart of another robot control method provided in an embodiment of the present application;

FIG. 2B is a schematic diagram of a human root model and a part model provided by an embodiment of the present application;

FIG. 3 is a flow chart of another robot control method provided in an embodiment of the present application;

fig. 4 is a structural diagram of a robot control device according to an embodiment of the present application;

fig. 5 is a structural diagram of a robot according to a fifth embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart of a robot control method provided in an embodiment of the present application, where the method is applied to a scenario of elevator taking control of a robot. The method may be performed by a robot control device, which may be implemented in software and/or hardware and which is specifically configured in an electronic device. The electronic device may be located inside the robot or may exist independently of the robot.

Referring to fig. 1, a robot control method includes:

s110, after an elevator door of an area where the robot is located is opened, the robot is controlled to move and rotate according to a preset path, and target images of at least two view field angles in the elevator car are collected.

Wherein, the car is the structure that supplies the user to take in the elevator. The area in which the robot is located includes floors in order to accurately position the elevator doors. When at least two elevator zones are included in one floor, the zone where the robot is located also includes an elevator zone identification.

The robot is provided with an image acquisition device which is used for acquiring images inside the elevator car in real time or at regular time after an elevator door of an area where the robot is located is opened, and taking the obtained images as target images to be used as a basis for determining the number of people taking the elevator. Illustratively, the image capturing device may be a camera that performs target image capturing at a set frequency. For example, the set frequency may be 200 frames/second.

It can be understood that target image acquisition is carried out through the image acquisition device arranged on the robot, so that the multiplexing of the image acquisition device of the robot is realized, the image acquisition device is not required to be additionally arranged in the car, and the hardware cost investment is reduced. Meanwhile, the robot determines the number of the subsequent target elevator passengers based on the acquired target image without the help of third-party data, so that the reliability of the determination result of the number of the target elevator passengers is higher, and the condition that the determination result of the number of the target elevator passengers is inaccurate due to the fact that a third-party data source is false or data transmission is safe is avoided. In addition, an image acquisition device is additionally arranged in the elevator car, image data needs to be transmitted to the robot in a communication mode, when wireless communication signals are unstable, the problem that the robot runs unsmoothly due to the fact that the image data cannot be transmitted in time exists, and therefore the acquisition of target images cannot be achieved in real time. Moreover, additionally set up image acquisition device or other sensors in the car, will reform transform elevator equipment, to the certain destructiveness of elevator equipment part, it is comparatively difficult to fall to the ground.

The preset path can be a safe driving path of a preset robot in a set area of an elevator door, so that the robot is controlled to move and rotate in the set area of the elevator door according to the preset path, images inside a car can be collected from at least two view field angles, target images of multiple view field angles are obtained, the richness and the comprehensiveness of the target images are improved, and the influence of people shielding on the accuracy of the follow-up target elevator riding number determination result is avoided.

For example, a call request can be sent to the elevator central control terminal by the robot, so that the elevator central control terminal controls the car to stop and open the elevator door corresponding to the target elevator taking area according to the target elevator taking area in the call request.

In an optional embodiment, the elevator door of the area where the robot is located is opened, the opening state of the elevator door can be determined by a detection device arranged near the elevator door in the area where the robot is located, and an opening notice is sent to the robot after the elevator door is opened, so that the robot is triggered to perform image acquisition. The detection device may be implemented by at least one device in the prior art, and may be an infrared detection device, for example.

In another optional embodiment, the elevator central control end can also determine the time information of the elevator car reaching the area where the robot is located, and after the time information is accumulated for setting a time interval, the start notification is sent to the robot so as to trigger the robot to perform image acquisition.

Because the detection device is additionally arranged, the hardware cost is increased, and the opening notification sending mode is carried out through the elevator central control end, the result accuracy depends on the reliability of the elevator central control end, meanwhile, bandwidth resources are required to be occupied for data transmission, and the transmission time delay and the data safety existing in the transmission process have certain influence on the opportunity of carrying out image acquisition on the robot. In order to improve the accuracy of the image acquisition opportunity and reduce unnecessary resource loss, in yet another optional embodiment, the robot may further detect whether an obstacle exists in the elevator door direction through the obstacle detection module, and in the absence of the obstacle, the elevator door may be considered to be opened, otherwise, the elevator door may be considered to be closed.

The robot determines the number of people taking the elevator from the target image, and the definition of the target image directly influences the accuracy of the determination result of the number of people taking the elevator from the target. The illumination intensity of the target image acquisition environment will directly affect the definition of the acquired target image. Therefore, the working state of the light supplementing unit in the robot can be controlled through the definition of the target image, so that the definition of the subsequently acquired target image is improved.

For example, after the target image is acquired for the first time in the area where the robot is located, the definition of the target image can be identified; if the definition meets the definition condition, setting the working state of the light supplementing unit to be an open state; otherwise, setting the working state of the light supplementing unit to be a closing state. The definition condition may be set by a definition region, and the definition region may be set or adjusted by a technician according to needs or experience values.

And S120, sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking results.

In an optional embodiment, people are calibrated and tracked in sequence for each target image, and the number of the target elevator passengers in the elevator car is determined according to the calibration and tracking results, wherein the calibration can be performed on newly added people in the target image and the tracking can be performed on the calibrated people in sequence; determining the current elevator taking number in the elevator car according to the calibration result and the tracking result in each target image; directly taking the current number of people taking the elevator as the target number of people taking the elevator.

Since the elevator is taken in and out by the user, the number of people taking the elevator is directly used as the target number of people taking the elevator, and a certain deviation exists, so that the target number of people taking the elevator can be determined according to the number of people taking the elevator and the current number of people taking the elevator.

It can be understood that the situation that the calibrated target in the previous target image is deviated may occur in the process of changing the angle of view, so that the target images at different angles of view can be collected in the process of moving the angle of view by controlling the movement and rotation of the robot, so that the addition of a newly added target or the tracking of the calibrated person can be performed according to the characteristics, the determination of the current elevator taking number is further realized, and the determination accuracy of the current elevator taking number is improved.

And S130, controlling the robot to take the elevator according to the number of the target elevator taking people.

For example, if the number of the target elevator taking people is larger than the set number of people threshold, the robot is forbidden to enter the elevator car, namely the robot is forbidden to take the elevator; and if the target elevator taking number is not more than the set number threshold, controlling the robot to run to the elevator car, namely allowing the robot to take the elevator.

According to the embodiment of the application, after an elevator door of an area where the robot is located is opened, the robot is controlled to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car; sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator taking people in the elevator car according to the calibration and tracking result; and controlling the robot to take the elevator according to the number of the target elevator taking people. According to the technical scheme, the robot obtains the target image and determines the number of the elevator passengers, so that the accuracy of the determination result is improved. Meanwhile, the number of people taking the elevator based on the target replaces the mode of whether the elevator car is empty or not to carry out elevator taking control on the robot, so that the robot is prevented from being influenced by overlong waiting time, and meanwhile, the influence of too much invalid elevator taking on the operation efficiency of the elevator is reduced, and the operation efficiency of the robot and the elevator is improved.

Example two

Fig. 2A is a flowchart of another robot control method according to an embodiment of the present application, which is based on the above technical solutions, and refines the operation of "determining the target number of passengers in the car according to the target image" into "determining the current number of passengers in the car according to the target image; and determining the target elevator taking number according to the elevator in-out counting result and the current elevator taking number so as to improve the accuracy of the target elevator taking number determining result. In the present application, reference is made to the foregoing embodiments for the purpose of illustration and description.

Referring to fig. 2A, a robot control method includes:

s210, after an elevator door of an area where the robot is located is opened, the robot is controlled to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car.

And S220, sequentially calibrating and tracking people for each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking results.

In an optional embodiment, the person calibration and tracking are sequentially performed on each target image, and the number of the target elevator passengers in the elevator car is determined according to the calibration and tracking result, which may be: and respectively identifying the number of the human faces in each target image through a human face detection model, so that the current elevator taking number in the elevator car is determined according to the number of the human faces. The face detection model can be realized based on a machine learning model.

In another optional embodiment, the person calibration and tracking are sequentially performed on each target image, and the number of the target elevator passengers in the elevator car is determined according to the calibration and tracking result, which may be: and respectively identifying the number of human faces in each target image in an edge detection mode, so as to determine the current elevator taking number in the elevator car according to the number of the human faces.

In yet another alternative embodiment, the current number of passengers in the car is determined according to the target image, which may be: determining a Histogram of Oriented Gradient (HOG) of the target image to obtain target feature data; and processing the target characteristic data by adopting a classification model to obtain pedestrian and non-pedestrian category predictions, and carrying out character calibration and subsequent tracking on the predicted object, thereby determining the current elevator taking number according to calibration and tracking results. The classification model may be implemented based on a Machine learning model, such as a Support Vector Machine (SVM).

Because the target image is an image acquired by the robot outside the car, people are shielded, the current number of people taking the elevator is determined by the method, and the accuracy of the determination result is poor. In order to improve the accuracy of the current elevator passenger number determination result, in yet another optional embodiment, a Deformable Part Model (DPM) may be further introduced during the feature extraction process of the target image, so as to adapt to situations such as occlusion of elevator passengers and human body posture deformation.

It should be noted that the DPM algorithm adopts the improved HOG feature, SVM classifier and sliding window detection concept, adopts a multi-Component (Component) strategy for the multi-view problem of the target to be detected (elevator passenger) in the target image, and adopts a Component model strategy based on a graph Structure (graphical Structure) for the deformation problem of the target itself. Further, the number of passengers is automatically determined by multi-instance Learning (multi-instance Learning) using the model type to which the sample belongs, the position of the component model, and the like as Latent variables (Latent Variable).

Wherein the DPM model comprises a root model, at least two component models, and a loss of separation of the component models with respect to the root model.

See figure 2B for a schematic illustration of the human root model, part model and loss of separation.

The root model in fig. 2B (a) belongs to a global template which is rough and covers the entire target, and is also called a root filter.

The part model in fig. 2B (B) belongs to a local template with a high resolution and covering a local region (e.g., a head, an arm, a leg, etc.) in the target, and is also called a part filter. The human target is divided into 6 parts, head, two upper limbs, two lower limbs and feet. The resolution of the part model is higher than the resolution of the root model. For example, the resolution of the part model is twice the resolution of the root model. To reduce the model complexity, both the root model and the part model are axisymmetric.

The deviation loss of the component model in (c) of fig. 2B with respect to the root model can be obtained by extracting HOG features from existing training samples of the human body, limbs, and the like, and then training the extracted HOG features by an SVM classifier. The lighter regions indicate that the deviation loss cost is larger, and the deviation loss of the rational position of the part model is 0.

Exemplarily, the target image can be processed according to a preset root model and each component model to obtain target response data; and according to the target response data, people are calibrated and tracked for the target image, so that the current elevator taking number in the elevator car is determined according to the calibration and tracking result.

It can be understood that the target image is processed by the root model and the component model, so that target detection can be converted into detection and recognition of different local structures (head, arms, legs and the like) in the target, and the influence of deformation conditions caused by various postures of the target in the target image, such as squatting, sitting, standing and the like, on the detection result is eliminated.

In a specific implementation manner, processing a target image according to a preset root model and each component model to obtain target response data includes: extracting the features of the target image to obtain initial feature data; processing the initial characteristic data according to the root model and each component model respectively to obtain initial response data; and generating target response data according to the initial response data.

Specifically, feature extraction is carried out on a target image to obtain initial global feature data; performing up-sampling on a target image, and performing feature extraction on an up-sampling result to obtain initial local feature data; performing convolution processing on the initial global feature data according to the root model to obtain target global response data; performing convolution processing on the initial local characteristic data by adopting each component model to obtain each initial local response data; respectively carrying out down-sampling on each initial local response data to obtain target local response data so as to enable the resolution of each target local response data to be the same as that of the target global response data; and determining the weighted average of the target global response data and each target local response data to obtain target response data.

For example, the feature extraction is performed on the target image, and the root model may be used to perform convolution processing on the target image, so as to obtain initial global feature data. Correspondingly, feature extraction is carried out on the up-sampling result, and the root model can be adopted to carry out convolution processing on the up-sampling result, so that initial local feature data can be obtained.

In one specific implementation, the person calibration and tracking of the target image according to the target response data includes: processing the target response data based on the classification model to obtain a classification result; and calibrating the newly added people and tracking the calibrated people according to the target position information corresponding to the classification result. According to the information of the identified model, the information comprises position information, and whether the person is a person can be determined based on the position information of each part of the human body by identifying the classification result, so that the number of people is determined, and the problem of counting the number of people by mistake is avoided. Therein, the classification model may be implemented based on a machine learning model, such as an SVM model.

Specifically, according to the target response data, determining each local area in the target image, and determining whether each local area belongs to the same target; tracking the same target and calibrating different targets; and taking the finally determined target number as the current elevator taking number.

Specifically, the DPM improved HOG cancels the Block (Block) in the original HOG and only retains the Cell (Cell). In normalization, a region composed of the current cell and 4 cells around the current cell is directly normalized. When calculating the gradient direction, a combination of signed (0-360 °) and unsigned (0-180 °) gradient directions may be calculated. Taking an 8 × 8 unit in the target image as an example, after normalization and truncation of the relative neighborhood, 4 unit groups are obtained, and the corresponding obtained signed gradient direction histogram is a 4 × 18 dimensional matrix, and the unsigned gradient direction histogram is a 4 × 9 dimensional matrix. Accumulating and summing the 4 x 18-dimensional signed gradient direction histograms according to columns to obtain 18-dimensional vectors; accumulating and summing the unsigned gradient direction histograms of 4 multiplied by 9 dimensions according to columns to obtain 9-dimensional vectors; accumulating and summing the unsigned gradient direction histograms of 4 multiplied by 9 dimensions according to rows to obtain 4-dimensional vectors; and combining the obtained 18-dimensional vector, 9-dimensional vector and 4-dimensional vector to obtain the feature vector of the unit.

For example, local areas including at least one of the head, the upper limbs, the lower limbs, the feet and the like are determined through the target response data, and whether each local area belongs to the same target (human body) is determined according to the target position information of each local area, so that the current elevator riding number in the elevator car is determined.

And S230, determining the target elevator taking number according to the elevator in-out counting result and the current elevator taking number.

For example, the input elevator statistical result can be obtained by a counting device arranged in the car, near the elevator door or in the robot, and used for counting the number of people getting in and out of the elevator to obtain the elevator getting in and out statistical result; and updating the current elevator taking number according to the statistical result of the elevator entering and exiting, and taking the updated current elevator taking number as the target elevator taking number.

And S240, controlling the robot to take the elevator according to the number of the target elevator taking people.

According to the method and the device, the determination operation of the target elevator taking number is refined into the determination of the current elevator taking number in the elevator car according to the target image, and the target elevator taking number is determined according to the elevator entering and exiting statistical result and the current elevator taking number, so that the problem that the elevator taking decision of the robot is influenced due to the fact that the target elevator taking number is inaccurate due to the fact that people in the elevator car change after the current elevator taking number is determined is avoided.

EXAMPLE III

Fig. 3 is a flowchart of a robot control method according to a third embodiment of the present application, which is based on the above technical solutions, and refines the operation "determining the target number of passengers in the car according to the calibration and tracking result" into "determining the current number of passengers in the car according to the calibration and tracking result; and determining the target elevator taking number in the elevator car according to the current elevator taking number and the historical elevator taking number so as to reduce the hardware cost for determining the target elevator taking number. In the present application, reference is made to the foregoing embodiments for the purpose of illustration and description.

Referring to fig. 3, a flow chart of a robot control method includes:

and S310, after an elevator door of an area where the robot is located is opened, controlling the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car.

S320, sequentially calibrating and tracking people of each target image, and determining the current elevator taking number in the elevator car according to the calibration and tracking result

The determination process of the number of people taking the elevator at present can be referred to the foregoing embodiment, and is not described herein again.

And S330, determining the target elevator taking number in the elevator car according to the current elevator taking number and the historical elevator taking number.

The historical elevator taking number can be the current elevator taking number corresponding to each collected target image after the elevator door is opened and before the current elevator taking number is determined.

For example, the target number of elevator passengers in the car is determined according to the current number of elevator passengers and the historical number of elevator passengers, and may be: if the current elevator taking number and the historical elevator taking number tend to be stable, taking the current elevator taking number as a target elevator taking number; and if not, taking the current elevator taking number as the historical elevator taking number, and re-determining the current elevator taking number according to the re-collected target image.

The current number of people taking the elevator and the historical number of people taking the elevator are stable in area, and the current number of people taking the elevator and the historical number of people taking the elevator are different from each other, so that a threshold value can be set. The set threshold value may be determined by a technician as needed or an empirical value, and may be 0, for example. The number of the adjacent historical elevator passengers can be the number of the adjacent historical elevator passengers with the set number, or the number of the historical elevator passengers determined in the set historical time period in which the number of the current elevator passengers is adjacent. Wherein the set number or set historical time period may be set by a technician as desired or empirically, or adjusted through a number of experiments. For example, the set number may be 2, and the set history period may be 3 seconds.

And S340, controlling the robot to take the elevator according to the number of the target elevator taking people.

The number of the target elevator taking persons is determined, and the operation is refined into the current number of the target elevator taking persons in the elevator car according to the calibration and tracking results; the number of the target elevator taking persons in the lift car is determined according to the current number of the elevator taking persons and the historical number of the elevator taking persons, so that the number of the target elevator taking persons in the lift car can be accurately determined only by performing software processing through a robot instead of additionally arranging a technical device, and the hardware cost is considered while the accuracy of the result of determining the number of the target elevator taking persons is improved.

Example four

Fig. 4 is a structural diagram of a robot control device according to an embodiment of the present application, where the device is applied to a scenario of robot elevator taking control. The apparatus may be implemented in software and/or hardware and is embodied in an electronic device. The electronic device may be located inside the robot or may exist independently of the robot.

Referring to fig. 4, a robot control apparatus includes: a target image acquisition module 410, a target elevator taking number determination module 420 and an elevator taking control module 430. Wherein the content of the first and second substances,

the target image acquisition module 410 is used for controlling the robot to move and rotate according to a preset path after an elevator door of an area where the robot is located is opened so as to acquire target images of at least two view field angles in the elevator car;

the target elevator taking number determining module 420 is used for sequentially calibrating and tracking people of each target image and determining the target elevator taking number in the elevator car according to the calibration and tracking result;

and an elevator taking control module 430, configured to control elevator taking of the robot according to the target number of elevator taking persons.

According to the embodiment of the application, after the elevator door of the area where the robot is located is opened, the target elevator taking number determining module controls the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car; sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator taking people in the elevator car according to the calibration and tracking result; and the elevator taking control module is used for controlling the robot to take the elevator according to the number of the target elevator taking people. According to the technical scheme, the robot obtains the target image and determines the number of the elevator passengers, so that the accuracy of the determination result is improved. Meanwhile, the number of people taking the elevator based on the target replaces the mode of whether the elevator car is empty or not to carry out elevator taking control on the robot, so that the robot is prevented from being influenced by overlong waiting time, and meanwhile, the influence of too much invalid elevator taking on the operation efficiency of the elevator is reduced, and the operation efficiency of the robot and the elevator is improved.

In an optional embodiment, the target elevator boarding population determining module 420 includes:

the target response data determining unit is used for processing the target image according to a preset root model and each component model to obtain target response data;

and the person calibration and tracking unit is used for calibrating and tracking the person of the target image according to the target response data.

In an optional embodiment, the target response data determining unit includes:

an initial feature data obtaining subunit, configured to perform feature extraction on the target image to obtain initial feature data;

an initial corresponding data obtaining subunit, configured to process the initial feature data according to the root model and each component model, respectively, to obtain initial response data;

and the target response data generating subunit is used for generating the target response data according to each initial response data.

In an alternative embodiment, the person targeting and tracking unit includes:

a classification result obtaining subunit, configured to process the target response data based on a classification model to obtain a classification result;

and the person calibration and tracking subunit is used for calibrating the newly added person and tracking the calibrated person according to the target position information corresponding to the classification result.

In an alternative embodiment, the target image acquisition module 410 includes:

the current elevator taking number determining unit is used for determining the current elevator taking number in the elevator car according to the calibration and tracking result;

and the target elevator taking number determining unit is used for determining the target elevator taking number in the elevator car according to the current elevator taking number and the historical elevator taking number.

In an optional embodiment, the target elevator boarding population determining unit includes:

the target elevator taking number determining subunit is used for taking the current elevator taking number as the target elevator taking number if the current elevator taking number and the historical elevator taking number tend to be stable; and if not, taking the current elevator taking number as the historical elevator taking number, and re-determining the current elevator taking number according to the re-collected target image.

In an optional embodiment, the apparatus further comprises:

and the call request sending module is used for sending a call request to an elevator central control end so that the elevator central control end controls the elevator car to stop and open the elevator door corresponding to the target elevator taking area according to the target elevator taking area in the call request.

In an optional embodiment, the target elevator boarding population determining unit includes:

the current elevator taking number determining subunit is used for determining the current elevator taking number in the elevator car according to the target image;

and the target elevator taking number determining subunit is used for determining the target elevator taking number according to the elevator entering and exiting statistical result and the current elevator taking number.

In an optional embodiment, the apparatus further comprises:

and the light supplementing unit control module is used for controlling the working state of the light supplementing unit in the robot according to the definition of a target image acquired in the area where the robot is located for the first time.

The robot control device can execute the robot control method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the robot control method.

EXAMPLE five

Fig. 5 is a structural diagram of a robot according to a fifth embodiment of the present application. FIG. 5 illustrates a block diagram of an exemplary robot 512 suitable for use in implementing embodiments of the present application. The robot 512 shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 5, the robot 512 is in the form of a general purpose computing device. The components of the robot 512 may include, but are not limited to: one or more processors or processing units 516, a system memory 528, and a bus 518 that couples the various system components including the system memory 528 and the processing unit 516.

Bus 518 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

The robot 512 typically includes a variety of computer system readable media. These media may be any available media that can be accessed by the robot 512 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 528 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 530 and/or cache memory 532. The bot 512 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 534 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 518 through one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

A program/utility 540 having a set (at least one) of program modules 542, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in, for example, the memory 528, each of which examples or some combination may include an implementation of a network environment. The program modules 542 generally perform the functions and/or methods of the embodiments described herein.

The robot 512 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), with one or more devices that enable a user to interact with the robot 512, and/or with any devices (e.g., network card, modem, etc.) that enable the robot 512 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 522. Also, the robot 512 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 520. As shown, the network adapter 520 communicates with the other modules of the robot 512 via a bus 518. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the robot 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 516 executes various functional applications and data processing by executing at least one of other programs stored in the system memory 528, for example, to implement the robot control method provided in the embodiment of the present application.

EXAMPLE six

The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program, when executed by a processor, implements a robot control method provided in any embodiment of the present application, and is executed by a robot, where the method includes: after an elevator door of an area where the robot is located is opened, controlling the robot to move and rotate according to a preset path so as to acquire target images of at least two view field angles in the elevator car; sequentially carrying out person calibration and tracking on each target image, and determining the number of the target elevator passengers in the elevator car according to the calibration and tracking result; and controlling the robot to take the elevator according to the number of the target elevator taking people.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.