阅读说明：本技术 在slam地图中重新定位移动载具的方法及移动载具 (Method for repositioning mobile carrier in SLAM map and mobile carrier ) 是由 洪士哲 陈豪宇 于 2020-08-11 设计创作，主要内容包括：本公开提出一种在一实时定位与地图绘制(SLAM)地图中重新定位一移动载具的方法及移动载具,该方法用于在一区域中移动的移动载具中,包括：在一初始时间点建立对应区域的SLAM地图；通过一非SLAM定位装置在SLAM地图上检测移动载具的一第一位置轨迹及一第一方位轨迹；通过一SLAM定位装置检测移动载具在一第一时间戳及一第二时间戳之间的一迷失几率；判断是否满足一条件；以及当在一目前时间点满足上述条件时,更新SLAM地图为对应目前时间点的一新SLAM地图及更新移动载具位于新SLAM地图中的一定位信息。(The present disclosure provides a method for relocating a mobile vehicle in a real time positioning and mapping (SLAM) map and the mobile vehicle, the method being used in the mobile vehicle moving in an area, comprising: establishing an SLAM map of a corresponding area at an initial time point; detecting a first position track and a first orientation track of the mobile carrier on the SLAM map through a non-SLAM positioning device; detecting a loss probability of the mobile carrier between a first time stamp and a second time stamp by a SLAM positioning device; judging whether a condition is met; and when the condition is met at a current time point, updating the SLAM map into a new SLAM map corresponding to the current time point and updating a positioning message of the mobile carrier in the new SLAM map.)

1. A method of repositioning a mobile vehicle in a real-time positioning and mapping map, for use in the mobile vehicle moving in an area, comprising:

establishing a real-time positioning and mapping map corresponding to the area using real-time positioning and mapping at an initial time point;

detecting a first position track and a first orientation track of the mobile carrier on the real-time positioning and mapping map through a non-real-time positioning and mapping positioning device;

detecting a loss probability of the mobile carrier between a first time stamp and a second time stamp through a positioning device for real-time positioning and mapping;

judging whether a condition is met; and

when the condition is met at a current time point, updating the map of the real-time positioning and mapping into a new map of the real-time positioning and mapping corresponding to the current time point and updating positioning information of the mobile carrier in the new map of the real-time positioning and mapping;

wherein the above condition is one of the following:

the location trajectory or the orientation trajectory is not between a first range and the loss probability is not between a second range; and

the chance of getting lost is not between the second range.

2. The method of claim 1, wherein updating the real-time position and map mapped map to the new real-time position and map mapped map and updating the position information further comprises:

updating the real-time positioning and mapping map to the new real-time positioning and mapping map and updating the positioning information using a position and an orientation of the mobile carrier at the current time point detected by the non-real-time positioning and mapping positioning device.

3. The method of claim 1, wherein prior to updating the real-time position and map mapped map to the new real-time position and map mapped map and updating the position information, the method further comprises:

calculating a first trustable value of the mobile carrier at the current time point through the positioning device for non-real-time positioning and mapping;

calculating a second trustable value of the mobile carrier at the current time point through the positioning device for real-time positioning and mapping; and

when the first trustworthy value or the second trustworthy value is higher than a threshold, updating the real-time positioning and mapping map to the new real-time positioning and mapping map corresponding to the current time point and updating the positioning information of the mobile vehicle in the new real-time positioning and mapping map.

4. The method of claim 3, wherein the first confidence value and the second confidence value are a mean function.

5. The method of claim 1, wherein the loss probability is a second location trajectory, a second orientation trajectory, or a congruent difference between the first timestamp and the second timestamp of the moving vehicle moved between the first timestamp and the second timestamp.

6. The method of claim 1 wherein said probability of loss is a mean function.

7. The method of claim 1, wherein the real-time position and map positioning device updates the real-time position and map to the new real-time position and map at the current time point in real time.

8. The method of claim 1, wherein said non-real time positioning and mapping positioning device occasionally updates said real time positioning and mapping map to said new real time positioning and mapping map.

9. The method of claim 1, wherein the first timestamp and the second timestamp are consecutive timestamps.

10. The method of claim 1, wherein the first timestamp and the second timestamp are non-consecutive timestamps.

11. A mobile vehicle for moving in an area, comprising:

a computing device to establish a real-time positioning and mapping map corresponding to the area using real-time positioning and mapping at an initial point in time;

a positioning device for non-real-time positioning and mapping, connected to the computing device, for detecting a first position track and a first orientation track of the mobile vehicle on the real-time positioning and mapping map; and

a real-time positioning and mapping positioning device, connected to the computing device, for detecting a loss probability of the mobile vehicle between a first timestamp and a second timestamp:

wherein the computing device determines whether a condition is satisfied; when the condition is met at a current time point, updating the real-time positioning and mapping map into a new real-time positioning and mapping map corresponding to the current time point and updating positioning information of the mobile carrier in the new real-time positioning and mapping map;

wherein the above condition is one of the following:

the location trajectory or the orientation trajectory is not between a first range and the loss probability is not between a second range; and

the chance of getting lost is not between the second range.

12. The mobile vehicle of claim 11, wherein updating the real-time position and map mapped map to the new real-time position and map mapped map and updating the position information further comprises:

updating the real-time positioning and mapping map to the new real-time positioning and mapping map and updating the positioning information using a position and an orientation of the mobile carrier at the current time point detected by the non-real-time positioning and mapping positioning device.

13. The mobile vehicle of claim 11, wherein prior to updating the real-time position and mapping map to the new real-time position and mapping map and updating the position information, the mobile vehicle further performs:

the non-real-time positioning and mapping positioning device calculates a first credible value of the mobile carrier at the current time point;

the positioning device for real-time positioning and mapping calculates a second credible value of the mobile carrier at the current time point; and

when the first trustworthy value or the second trustworthy value is higher than a threshold, the computing device updates the real-time positioning and mapping map to the new real-time positioning and mapping map corresponding to the current time point and updates the positioning information of the mobile vehicle in the new real-time positioning and mapping map.

14. The mobile vehicle of claim 13, wherein the first trustworthy value and the second trustworthy value are a mean function.

15. The mobile vehicle of claim 11, wherein the loss probability is a second location trajectory, a second orientation trajectory, or a congruent difference between the first timestamp and the second timestamp of the real-time positioning and mapping map moved by the mobile vehicle between the first timestamp and the second timestamp.

16. The mobile carrier of claim 11, wherein the probability of getting lost is a mean function.

17. The mobile vehicle of claim 11, wherein the real-time position and map location device updates the real-time position and map to the new real-time position and map at the current point in time.

18. The mobile vehicle of claim 11, wherein the non-real-time positioning and mapping positioning device occasionally updates the real-time positioning and mapping map to the new real-time positioning and mapping map.

19. The mobile vehicle of claim 11, wherein the first timestamp and the second timestamp are consecutive timestamps.

20. The mobile vehicle of claim 11, wherein the first timestamp and the second timestamp are non-contiguous timestamps.

Technical Field

The present disclosure relates to a method for repositioning a mobile vehicle and a mobile vehicle, and more particularly, to a method for repositioning a mobile vehicle in a map of real-time positioning and Mapping (SLAM) and a mobile vehicle.

Background

As a system that enables a mobile robot to map its environment and maintain working data of its position in the map, real-time positioning and mapping are both accurate and versatile. Its reliability and adaptability to various applications make it a useful element to give the robot a certain degree of autonomy.

Currently, SLAM technology is to locate the position of a mobile robot and map it using probability calculation. Since a more precise position or orientation is required using this technique, the mobile robot cannot reposition itself in the currently rendered SLAM map (i.e., the mobile robot becomes lost) once it encounters some complex or more varied environment. The map and positioning information previously established by the mobile robot will also be totally invalidated.

Therefore, there is a need for a method of relocating a mobile vehicle in a SLAM map and a mobile vehicle to improve the above problems.

Disclosure of Invention

The following disclosure is illustrative only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features, other aspects, embodiments, and features will be apparent by reference to the drawings and the following detailed description. That is, the following disclosure is provided to introduce concepts, points, benefits and novel and non-obvious technical advantages described herein. Selected, but not all, embodiments are described in further detail below. Accordingly, the following disclosure is not intended to identify essential features of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter.

It is therefore a primary objective of the present disclosure to provide a method for relocating a mobile vehicle in a SLAM map and the mobile vehicle, so as to improve the above disadvantages.

The present disclosure provides a method for repositioning a mobile vehicle in a real-time positioning and mapping map, for use in the mobile vehicle moving in an area, comprising: using SLAM to establish the SLAM map corresponding to the area at an initial time point; detecting a first position track and a first orientation track of the mobile vehicle on the SLAM map through a non-SLAM positioning device; detecting a loss probability of the mobile carrier between a first time stamp and a second time stamp by a SLAM positioning device; judging whether a condition is met; when the condition is met at a current time point, updating the SLAM map into a new SLAM map corresponding to the current time point and updating positioning information of the mobile carrier in the new SLAM map; wherein the above condition is one of the following: the position track or the orientation track is not between a first range and the loss probability is not between a second range; and the loss probability is not between the second range.

In some embodiments, the step of updating the SLAM map to the new SLAM map and the positioning information further comprises: updating the SLAM map to the new SLAM map and updating the positioning information using a position and a location of the mobile vehicle at the current time point detected by the non-SLAM positioning device.

In some embodiments, before updating the SLAM map to the new SLAM map and updating the positioning information, the method further comprises: calculating a first trustworthy value of the mobile vehicle at the current time point by the non-SLAM positioning device; calculating a second trustable value of the mobile carrier at the current time point by the SLAM positioning device; and updating the SLAM map to the new SLAM map corresponding to the current time point and updating the positioning information of the mobile vehicle in the new SLAM map when the first trustworthiness value or the second trustworthiness value is higher than a threshold.

In some embodiments, the first trustworthy value and the second trustworthy value are a mean function.

In some embodiments, the loss probability is a second location trajectory, a second bearing trajectory, or an overlapping difference between the first timestamp and the second timestamp of the mobile vehicle moving between the first timestamp and the second timestamp.

In some embodiments, the probability of getting lost is a mean function.

In some embodiments, the SLAM locating device updates the SLAM map to the new SLAM map at the current time point in real time.

In some embodiments, the non-SLAM locating device updates the SLAM map to the new SLAM map at irregular times.

In some embodiments, the first timestamp and the second timestamp are consecutive timestamps.

In some embodiments, the first timestamp and the second timestamp are discontinuous timestamps.

The present disclosure provides a mobile carrier for moving in an area, including: a computing device, which uses SLAM to establish the SLAM map corresponding to the area at an initial time point; a non-SLAM positioning device connected to the computing device for detecting a first position track and a first orientation track of the mobile vehicle on the SLAM map; and a SLAM locating device, connected to the computing device, for detecting a probability of the mobile carrier missing between a first time stamp and a second time stamp: wherein the computing device determines whether a condition is satisfied; when the condition is met at a current time point, updating the SLAM map into a new SLAM map corresponding to the current time point and updating positioning information of the mobile carrier in the new SLAM map; wherein the above condition is one of the following: the position track or the orientation track is not between a first range and the loss probability is not between a second range; and the loss probability is not between the second range.

Drawings

Fig. 1 schematically shows a mobile vehicle in the form of an unmanned vehicle installation equipped with a SLAM locating device and a non-SLAM locating device.

Fig. 2 is a diagram illustrating a SLAM map generated by a mobile vehicle according to data obtained by a SLAM locating device according to an embodiment of the disclosure.

Fig. 3 is a flowchart illustrating a method for repositioning a mobile vehicle in a real-time positioning and mapping map according to an embodiment of the disclosure.

Fig. 4 is a schematic diagram illustrating SLAM map overlay according to an embodiment of the disclosure.

FIG. 5 is a diagram showing an exemplary operating environment for implementing embodiments of the present disclosure.

The reference numerals are explained below:

100: mobile carrier

102: SLAM positioning device

104: non-SLAM positioning device

110: computing device

112: processor with a memory having a plurality of memory cells

114: memory device

1142: procedure for measuring the movement of a moving object

200: SLAM map

300: method flow chart

S305, S310, S315, S320, S325, S330: step (ii) of

410: SLAM map

420: SLAM map

500: computing device

510: bus line

512: memory device

514: processor with a memory having a plurality of memory cells

516: display element

518: I/O port

520: I/O element

522: power supply

Detailed Description

Aspects of the present disclosure are described more fully hereinafter with reference to the accompanying drawings. This disclosure may, however, be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the present disclosure is intended to encompass any aspect disclosed herein, whether alone or in combination with any other aspect of the present disclosure to achieve any aspect disclosed herein. For example, it may be implemented using any number of the apparatus or performing methods set forth herein. In addition, the scope of the present disclosure is more intended to cover apparatuses or methods implemented using other structures, functions, or structures and functions in addition to the aspects of the present disclosure set forth herein. It is to be understood that any aspect disclosed herein may be embodied by one or more elements of a claim.

The word "exemplary" is used herein to mean "serving as an example, instance, or illustration. Any aspect of the present disclosure or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects of the present disclosure or design. Moreover, like numerals refer to like elements throughout the several views, and the articles "a" and "an" include multiple references unless otherwise specified in the description.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between …" versus "directly between …," "adjacent" versus "directly adjacent," etc.).

The disclosed embodiments provide a method for relocating a mobile vehicle in a real-time location and Mapping (SLAM) map and a mobile vehicle, wherein another accurate non-SLAM location device is added to further solve the problem of location failure by using the SLAM location device.

Fig. 1 schematically shows a mobile vehicle 100 in the form of an unmanned vehicle installation equipped with a SLAM locating device 102 and a non-SLAM locating device 104. In the embodiment shown, the SLAM locating device 102 can be any suitable ranging sensor (e.g., a Laser (LiDAR), laser scanner (laser scanner), or sonar device) that can produce an accurate map. The non-SLAM location device 104 may be any Radio Frequency (RF) based location device (e.g., Ultra-wideband (UWB), Bluetooth (Bluetooth), LoRa, WiFi, etc. devices). Mobile vehicle 100 also includes computing device 110 having processor 112 and a memory 114 that may store program 1142.

The computing device 110 is a device capable of supporting various wireless access technologies, such as a mobile phone, a notebook computer, a smart phone, or a tablet computer. The computing device 110 is in communication with the SLAM locating device 102 and the non-SLAM locating device 104 (e.g., via a wireless communication interface), and is capable of storing and processing data related to using signals received by the SLAM locating device 102 and the non-SLAM locating device 104.

The memory 114 may store a SLAM map that is created by the processor 112 using SLAM for an area. The computing device 110 may further include other conventional features such as a user interface and a communication interface that allow it to exchange data with remote devices. In alternative embodiments, the signal from the at least one sensor may be transmitted to a remote computing device for processing, rather than being processed by the computing device 110.

In use, the mobile vehicle 100 travels along a road surface in an area and data relating to the scene surrounding the mobile vehicle 100 is captured by the SLAM locating device 102 and the non-SLAM locating device 104. Although the exemplary mobile vehicle 100 is an unmanned vehicle that travels along a road/ground, it should be appreciated that in alternative embodiments, the mobile vehicle 100 may be any type of device that can travel over (and not necessarily in contact with) a ground surface that may be scanned. Furthermore, in other embodiments, the SLAM locating device 102, the non-SLAM locating device 104, and the computing device 110 may not need to be mounted on the mobile vehicle 100, but may be included in, for example, a handheld navigation device.

The processor 112 is configured to process data received from the SLAM locating device 102 and the non-SLAM locating device 104 to attempt to locate the mobile vehicle 100 from existing map data. The processor 112 may use the data obtained from the SLAM locating device 102 to generate a SLAM map of the mobile vehicle 100 while it is running, as shown in fig. 2. Fig. 2 shows a SLAM map 200 generated by the mobile vehicle 100 according to data obtained by the SLAM locating device 102 according to an embodiment of the disclosure. As shown, the SLAM map 200 is composed of at least three data-1, 0, 1, where-1 represents an unknown area (e.g., a gray area in fig. 2), 0 represents a known area (e.g., a black area, a wall, or an obstacle in fig. 2), and 1 represents a blank area (e.g., a white area or a blank space in fig. 2).

It should be understood that the computing device 110 shown in FIG. 1 may be implemented via any type of computing device, such as the computing device 500 described with reference to FIG. 5, as shown in FIG. 5.

Fig. 3 is a flowchart 300 of a method for repositioning a mobile vehicle in a real-time positioning and Mapping (SLAM) map according to an embodiment of the disclosure. The method may be performed in a mobile carrier 100 moving over an area as shown in fig. 1.

In step S305, the mobile vehicle uses SLAM to establish the SLAM map corresponding to the area at an initial time point.

In step S310, the mobile vehicle detects a first position track and a first orientation track of the mobile vehicle on the SLAM map through a non-SLAM positioning device. More specifically, the non-SLAM locating device can record the position and the orientation of the mobile vehicle on the SLAM map in real time. All positions can be represented as a set of positions rf.p { { timestamp1, position1}, { timestamp2, position2}, … }, and all orientations can be represented as a set of orientations rf.a { { timestamp1, azrimuth1}, { timestamp2, azrimuth2}, … }. The first position track is formed by more than two continuous positions in the position set, and the first direction track is formed by more than two continuous directions in the direction set.

In step S315, the mobile vehicle detects a loss probability of the mobile vehicle between a first timestamp and a second timestamp through a SLAM positioning device, wherein the loss probability is a second location track, a second orientation track or an overlapping difference between the first timestamp and the second timestamp of the mobile vehicle moving between the first timestamp and the second timestamp.

More specifically, the SLAM locating device can record the position and the orientation of the mobile vehicle on the SLAM map in real time. All positions may be represented as a set of positions opt { { timing 1, position1}, { timing 2, position2}, … }, and all positions may be represented as a set of positions opt { { timing 1, azrimuth1}, { timing 2, azrimuth2}, … }. And the second position track is formed by two or any two continuous positions in the position set, and the first direction track is formed by two or any two continuous directions in the direction set. Further, a congruent difference of the SLAM map between the first timestamp and the second timestamp is a difference value of the SLAM map between the first timestamp and the second timestamp. For example, fig. 4 is a schematic diagram illustrating a SLAM map overlay according to an embodiment of the disclosure. SLAM map 410 is the SLAM map generated at time stamp t0, and SLAM map 420 is the SLAM map generated at time stamp ti. The SLAM locating device can calculate the probability values of the SLAM map 410 and the SLAM map 420 as a probability of losing the mobile vehicle between the time stamp t0 and the time stamp ti.

In S315 of fig. 3, the process of detecting the loss probability of the mobile vehicle between the first timestamp and the second timestamp by the SLAM positioning device can be represented by the following python program code, wherein the input parameters include at least the SLAM map _ t-1, the ranging sensor LiDAR input (i.e., the distance value between the mobile vehicle and the surrounding environment) at the previous time t-1, the position opt _ p _ t (x _ t, y _ t) and the orientation opt _ a _ t (a _ t) where the mobile vehicle is located at the time t, and "#" and the text on the right side thereof represent the annotations in the program.

In another embodiment, the probability of loss is a mean function, such as mean (p).

It should be noted that steps S310 and S315 are executed by the non-SLAM positioning device and the SLAM positioning device, respectively, at the same time.

Next, in step S320, the processor of the mobile carrier determines whether a condition is satisfied, wherein the condition is one of the following conditions: (1) the position track or the orientation track is not between a first range and the loss probability is not between a second range; and (2) the loss probability is not between the second range.

When the condition is satisfied at a current time point (yes in step S320), in step S325, the processor of the mobile vehicle updates the SLAM map to a new SLAM map corresponding to the current time point and updates a positioning information (e.g., a position and an orientation) of the mobile vehicle in the new SLAM map. More specifically, the processor of the mobile vehicle updates the SLAM map to the new SLAM map and updates the positioning information using a position and a location of the mobile vehicle at the current time point detected by the non-SLAM positioning device. For example, the position and orientation of the SLAM locating device of the mobile vehicle at the current time point i, which detects that the mobile vehicle is located on the SLAM map, are opt _ p _ i (x _ i, y _ i) and opt _ a _ i (a _ i), respectively, while the position and orientation of the non-SLAM locating device of the mobile vehicle at the current time point i, which detects that the mobile vehicle is located on the SLAM map, are rf _ p _ i (x _ i, y _ i) and rf _ a _ i (a _ i), respectively. When the above condition is satisfied at the current time point i, the processor of the mobile vehicle updates the position and the azimuth of the mobile vehicle detected by the SLAM positioning device to be located on the SLAM map from opt _ p _ i (x _ i, y _ i) and opt _ a _ i (a _ i) to rf _ p _ i (x _ i, y _ i) and rf _ a _ i (a _ i). The SLAM positioning device then updates the SLAM map into a new SLAM map corresponding to the current time point i by using the position and the directions rf _ p _ i (x _ i, y _ i) and rf _ a _ i (a _ i) of the mobile carrier on the SLAM map. After the SLAM positioning device updates the old SLAM map to the new SLAM map, the non-SLAM positioning device occasionally updates the old SLAM map to the new SLAM map.

In another embodiment, in step S325, before updating the SLAM map to the new SLAM map and updating the positioning information, the mobile vehicle may further calculate a first trustworthiness value of the mobile vehicle at the current time point through the non-SLAM positioning device and a second trustworthiness value of the mobile vehicle at the current time point through the SLAM positioning device. The trustworthiness value indicates whether the result of the relocation is trustworthy.

When the first credible value or the second credible value is higher than a threshold value, the mobile vehicle updates the SLAM map to the new SLAM map corresponding to the current time point and updates the positioning information of the mobile vehicle in the new SLAM map. And when the first credible value or the second credible value is not higher than the threshold value, the mobile vehicle abandons updating the SLAM map and the positioning information of the mobile vehicle.

More specifically, the first and second confidence values can be represented by python program code, wherein the input parameters are at least the position opt _ p _ t (x _ t, y _ t) and the orientation opt _ a _ t (a _ t) of the mobile vehicle on the SLAM map obtained by the SLAM locating device at time t, and the position rf _ p _ t (x _ t, y _ t) and the orientation and rf _ a _ t (a _ t) of the mobile vehicle on the SLAM map obtained by the non-SLAM locating device at time t, and the text on the right of "#" represents the annotation in the program.

In another embodiment, the first and second trustworthy values are a mean function, such as mean (p).

In addition, the processor 112 in the mobile vehicle 100 may also be integrated with the SLAM locating device 102 and the non-SLAM locating device 104. The processor 112 in the mobile carrier 100 can also execute the program 1142 in the memory 114 to present the actions and steps of the above embodiments, or other descriptions in the specification.

Thus, with the present disclosure of a method for relocating a mobile vehicle in a SLAM map and a mobile vehicle, the SLAM locating device of the mobile vehicle can relocate itself to the currently mapped SLAM map by using the location and bearing provided by another accurate non-SLAM locating device to improve the accuracy of the location in the SLAM map.

With respect to the described embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below. With specific reference to FIG. 5, FIG. 5 illustrates an exemplary operating environment for implementing embodiments of the present invention that may be generally considered a computing device 500. Computing device 500 is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing device 500 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

The present invention may be implemented in computer program code or machine-useable instructions, such as computer-executable instructions for program modules, executed by a computer or other machine, such as a personal digital assistant or other portable device. Generally, program modules include routines, programs, objects, components, data structures, etc., which refer to program code that performs particular tasks or implements particular abstract data types. The present invention may be implemented in a variety of system configurations, including portable devices, consumer electronics, general purpose computers, more specialized computing devices, and the like. The invention may also be implemented in a distributed computing environment, processing devices linked by a communications network.

Refer to fig. 5. Computing device 500 includes a bus 510 that directly or indirectly couples the following devices, a memory 512, one or more processors 514, one or more display elements 516, input/output (I/O) ports 518, input/output (I/O) elements 520, and an illustrative power supply 522. Bus 510 represents what may be one or more busses (e.g., an address bus, data bus, or combination thereof). Although the blocks of FIG. 5 are illustrated with lines for simplicity, in practice, the boundaries of the various elements are not specific, e.g., the presentation elements of the display device may be considered to be I/O elements; the processor may have a memory.

Computing device 500 typically includes a variety of computer-readable media. Computer readable media can be any available media that can be accessed by computing device 500 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer-readable media include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable-Programmable-Read-Only Memory (EEPROM), flash Memory or other Memory technology, Compact disk Read-Only Memory (CD-ROM), Digital Versatile Disk (DVD) or other optical disk storage, magnetic disks, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 500. Computer storage media itself does not include signals.

Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modular data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modular data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as audio, radio frequency, infrared and other wireless media. Combinations of the above are included within the scope of computer-readable media.

Memory 512 includes computer storage media in the form of volatile and non-volatile memory. The memory may be removable, non-removable, or a combination of the two. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. Computing device 500 includes one or more processors that read data from entities such as memory 512 or I/O elements 520. The display element 516 displays data indications to a user or other device. Exemplary display elements include display devices, speakers, printing elements, vibrating elements, and the like.

The I/O ports 518 allow the computing device 500 to be logically connected to other devices including I/O elements 520, some of which are built-in devices. Exemplary components include a microphone, joystick, game pad, satellite dish signal receiver, scanner, printer, wireless device, and the like. The I/O element 520 may provide a natural user interface for processing user-generated gestures, sounds, or other physiological inputs. In some examples, these inputs may be transmitted to a suitable network element for further processing. The computing device 500 may be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, and combinations of these systems, to detect and identify objects. In addition, the computing device 500 may be equipped with sensors (e.g., radar, light radar) to periodically sense the surrounding environment within a sensing range of the surrounding environment, generating sensor information indicative of its association with the surrounding environment. Still further, the computing device 500 may be equipped with an accelerometer or gyroscope to detect motion. The output of the accelerometer or gyroscope may be provided to computing device 500 for display.

Further, the processor 514 in the computing device 500 may also execute the programs and instructions in the memory 512 to present the actions and steps described in the above embodiments, or other descriptions in the specification.

Any particular order or hierarchy of steps for processes disclosed herein is by way of example only. Based upon design preferences, it should be understood that any specific order or hierarchy of steps in the processes may be rearranged within the scope of the disclosures made in this document. The accompanying method claims present elements of the various steps in a sample order, and are therefore not to be limited to the specific order or hierarchy presented.

The use of ordinal terms such as "first," "second," "third," etc., in the claims to modify an element does not by itself connote any priority, precedence, order of various elements, or order of steps performed by the method, but are used merely as labels to distinguish one element from another element having a same name (but for use of a different ordinal term).

Although the present disclosure has been described with reference to exemplary embodiments, it should be understood that various changes and modifications can be made therein by those skilled in the art without departing from the spirit and scope of the disclosure, and therefore, the scope of the disclosure should be limited only by the appended claims.