阅读说明：本技术 操作机动车辆的方法 (Method for operating a motor vehicle ) 是由 弗雷德里克·斯蒂芬 克里斯托夫·阿恩特德尔哈比尔 于 2021-05-12 设计创作，主要内容包括：本发明涉及一种用于操作机动车辆(4)的方法,该方法包括以下步骤：(S100)使用机动车辆(4)的环境传感器(6)获取环境数据集(UDS),(S200)将环境数据集(UDS)传输到云计算机(8),(S300)通过云计算机(8)生成用于补充环境数据集(UDS)的补充数据集(EDS),(S400)使用补充数据集(EDS)对环境数据集(UDS)进行补充,以便通过云计算机(8)生成补充环境数据集(UDS'),并且(S500)将补充环境数据集(UDS')传输到机动车辆(4)。(The invention relates to a method for operating a motor vehicle (4), comprising the following steps: (S100) acquiring an environmental data set (UDS) using an environmental sensor (6) of the motor vehicle (4), (S200) transmitting the environmental data set (UDS) to a cloud computer (8), (S300) generating a supplemental data set (EDS) for the supplemental environmental data set (UDS) by the cloud computer (8), (S400) supplementing the environmental data set (UDS) with the supplemental data set (EDS) so as to generate a supplemental environmental data set (UDS ') by the cloud computer (8), and (S500) transmitting the supplemental environmental data set (UDS') to the motor vehicle (4).)

1. A method for operating a motor vehicle (4), comprising the steps of:

(S100) acquiring an environmental data set (UDS) using an environmental sensor (6) of the motor vehicle (4),

(S200) transmitting the environment data set (UDS) to a cloud computer (8),

(S300) generating a complementary data set (EDS) for complementing the environmental data set (UDS) by means of the cloud computer (8),

(S400) supplementing the environmental data set (UDS) with the supplemental data set (EDS) to generate a supplemental environmental data set (UDS') by the cloud computer (8), and

(S500) transmitting the supplemental environment data set (UDS') to the motor vehicle (4).

2. The method according to claim 1, wherein the motor vehicle (4) transmits a state data set (ZDS) indicative of the state of the motor vehicle (4) to the cloud computer (8).

3. The method according to claim 1 or 2, wherein data (D) to be supplemented in the environmental dataset (UDS) (6) is identified and the cloud computer (8) evaluates the environmental dataset (UDS) when determining the supplementary dataset (EDS).

4. The method according to claim 1, 2 or 3, wherein the supplementary environmental data set (EDS') is a 4D data set.

5. Method according to one of the preceding claims, wherein the supplementary data set (EDS) is determined by evaluating data of other road users (V).

6. A computer program product for a motor vehicle (4) configured to perform the method according to one of claims 1 to 5.

7. A computer program product for a cloud computer (8) configured to perform the method according to one of claims 1 to 5.

8. A control device (10) for operating a motor vehicle (4), wherein the control device (10) is configured to acquire an environmental data set (UDS) using environmental sensors (6) of the motor vehicle (4), to transmit the environmental data set (UDS) to a cloud computer (8), and to receive and evaluate a supplemental environmental data set (UDS').

9. The control device (10) as claimed in claim 8, wherein the control device (10) is configured to transmit a state data set (ZDS) indicating the state of the motor vehicle (4) to the cloud computer (8).

10. The control device (10) according to claim 8 or 9, wherein the control device (10) is configured to identify data (D) in the environment data set (UDS) (6) that needs to be supplemented.

11. The control device (10) according to claim 7, 8 or 9, wherein the supplementary environment data set (UDS') is a 4D data set.

12. Motor vehicle (4) comprising a control device (10) according to one of claims 8 to 11.

13. A cloud computer (8), wherein the cloud computer (8) is configured to read in an environmental data set (UDS), generate a supplementary data set (EDS) for supplementing the environmental data set (UDS), supplement the environmental data set (UDS) with the supplementary data set (EDS) for generating a supplementary environmental data set (UDS '), and transmit the supplementary environmental data set (UDS') to a motor vehicle (4).

14. A cloud computer (8) according to claim 13, wherein the cloud computer (8) is configured to read in and evaluate a state data set (ZDS) indicative of the state of the motor vehicle (4).

15. A cloud computer (8) according to claim 13 or 14, wherein the cloud computer (8) is configured to determine the data (D) to be supplemented and to evaluate the environmental data set (UDS) when determining the supplementary data set (EDS).

16. A cloud computer (8) according to claim 13, 14 or 15, wherein the supplementary environment data set (EDS') is a 4D data set.

17. A cloud computer (8) according to one of claims 13 to 16, wherein the cloud computer (8) is configured to determine the supplementary data set (EDS) by evaluating data of other road users.

18. A system (2) comprising a motor vehicle (4) according to claim 12 and a cloud computer (8) according to one of claims 13 to 17.

Technical Field

The invention relates to a method for operating a motor vehicle.

Background

As the number of driver assistance systems has increased, more and more automatic driving functions have been provided, and the demand for high-precision sensor data has increased.

In the case of driver assistance systems (for example automatic parking assistance systems or automatic trajectory planning functions), it is important to clearly understand the environment of the motor vehicle before the driving operation is carried out (i.e. during the planning phase). And during the execution of the driving operation (i.e., while the operation is being performed).

Since these driving operations are performed automatically, i.e. without driver intervention, unreliable or undetected obstacles may cause a collision. For example, in the parking assist case, it can be a challenge for conventional ultrasonic sensors to detect low obstacles located under the bumper of a motor vehicle. The driver must therefore also monitor the environment of the motor vehicle and intervene if necessary.

In recent years, great progress has been made in sensor technology, in particular in cameras and lidar sensors and in the corresponding signal processing field, hardware components such as deep learning or GPUs. The cost of camera sensors has been significantly reduced, while the cost of high resolution lidar has remained very high. Furthermore, these sensor technologies require special mounting on the vehicle body. For example, to avoid malfunctions, it must be ensured that the camera lens remains clean and special control units and software are required, resulting in additional costs and additional power consumption.

US 8179241B 2, US 9403482B 2, US 10496890B 2 and US 2016/0339840 a1 respectively disclose systems and methods for supplementing environmental data acquired by an environmental sensor of a motor vehicle by compensating for shadows or gaps or blind spot areas in the sensor field to provide an improved, representative set of motor vehicle environmental data. To achieve this goal, sensor data provided by other road users, infrastructure or pedestrian smartphones is used.

US 9836056B 2 discloses a system and method in which a predictive data set is also provided which represents the environment of a motor vehicle in a brief future time window of a few seconds in duration. The prediction dataset is representative of a simulated 3D environment.

However, this occupies the computational resources of the motor vehicle and also increases the power consumption of the motor vehicle.

Therefore, it is necessary to find a method for remedying such a situation.

Disclosure of Invention

The object of the invention is achieved by a method for operating a motor vehicle, comprising the steps of:

acquiring an environmental data set using an environmental sensor of a motor vehicle;

transmitting the environmental dataset to a cloud computer;

generating, by the cloud computer, a supplemental dataset for supplementing the environmental dataset;

supplementing the environmental data set with a supplemental data set to generate, by the cloud computer, the supplemental environmental data set; and

the supplemented environmental data set is transmitted to the motor vehicle.

The environmental data set may be raw data or processed sensor data from environmental sensors (e.g., cameras, ultrasonic, radar, or lidar sensors) of the motor vehicle. The environmental data set is transmitted to the cloud computer over a wireless data transmission link (e.g., a 5G connection). The cloud computer may be a computer or a network of several computers, i.e. an IT infrastructure, which may be used, for example, over the internet. It typically includes memory space, computing power, or application software as a service. Thus, sensor data of the environmental dataset is outsourced to the cloud, fused with a supplemental dataset determined by the cloud computer, and the respective supplemental environmental dataset is again wirelessly transmitted to the motor vehicle. This can therefore save computational resources of the motor vehicle and can also use additional sensor data sources, thereby reducing the power requirements of the motor vehicle and improving the quality of the sensor data.

According to one embodiment, the motor vehicle transmits a state data set to the cloud computer, the state data set indicating a state of the motor vehicle. The state of the motor vehicle may be its position, direction of travel and/or speed of travel. In other words, the state data set comprises position data and/or driving direction data and/or driving speed data of the motor vehicle and thus allows a simple mapping of the additional sensor data to form the supplementary data set.

According to another embodiment, data to be supplemented is identified in the environmental dataset, and the cloud computer evaluates the environmental dataset when determining the supplemental dataset. In other words, on or on the motor vehicle side, a shadow or a gap or a blind spot region in the environmental sensor region is to be correspondingly identified. This can be done in the cloud computer of the cloud after the transfer to the cloud computer, or can also be done on the motor vehicle or on the motor vehicle side before the transfer to the cloud computer. This reduces the computing effort on the cloud computer or on the cloud side, since the data D to be supplemented can thus be determined in a particularly simple and resource-saving manner.

According to another embodiment, the supplementary ambient data set is a 4D data set. 4D is to be understood as the extension of the representation of an object in 3D space with coordinates x, y and z by another auxiliary dimension, here the temporal dimension. Thus, the 4D data set represents a period of time within a future time window. The data set may also be considered a predictive data set representative of the environment of the motor vehicle. Thus, the computational effort required to determine future behavior of other road users may be transferred to the cloud computer, further reducing the power requirements of the motor vehicle and further improving the quality of the sensor data.

According to another embodiment, the supplementary data set is determined by evaluating data of other road users. For this purpose, it is determined, for example, whether other road users are located in a predetermined area around the motor vehicle, and the data sets of the individual road users are downloaded from the motor vehicle in a manner similar to the environmental data sets. Thus, by exchanging the ambient data set between the motor vehicle and other road users, additional data is actually provided, thereby further improving the quality of the sensor data.

Furthermore, the invention comprises a computer program product for a motor vehicle, a computer program product for a cloud computer, a control device, a motor vehicle comprising the control device, a cloud computer and a system comprising the motor vehicle and the cloud computer.

Drawings

The invention will now be described with the aid of the accompanying drawings. The description is as follows:

FIG. 1: a schematic diagram of a system including a motor vehicle and a cloud computer;

FIG. 2: a schematic diagram of a first part of a method sequence for operating the system of figure 1;

FIG. 3: a schematic diagram of another portion of a method sequence for operating the system of FIG. 1;

FIG. 4: a schematic diagram of another portion of a method sequence for operating the system shown in FIG. 1;

FIG. 5: schematic diagram of another part of the method sequence for operating the system shown in fig. 1.

Detailed Description

Reference is first made to fig. 1.

A system 2 is shown that includes a motor vehicle 4 and a cloud computer 8.

The motor vehicle 4 and the cloud computer 8, as well as their respective components described below, may include hardware and/or software components for performing the tasks and functions described below.

In the present exemplary embodiment, the motor vehicle 4 is configured as a passenger vehicle and comprises at least one driver assistance system 16, for example an automatic parking assistance system or another driver assistance system with an automatic trajectory planning function. For example, according to one of SAE phases 1 to 5, motor vehicle 4 may be configured as an autonomous motor vehicle compliant with SAE (J3016).

The motor vehicle side transmitter 12 of the motor vehicle 4 allows the environment data set UDS to be transmitted to the cloud computer 8 over the wireless data link 42. Furthermore, the motor vehicle side transmitter 12 may receive a complementary environmental data set UDS' provided by the cloud computer 8 and transmitted over the wireless data link 42.

Furthermore, the motor vehicle 4 comprises an environmental sensor 6. The environment sensor 6 may be an actual sensor for acquiring environment data, e.g. a camera, an ultrasonic, radar or lidar sensor or a positioning system such as GPS. Furthermore, the environmental sensor 6 may also be a virtual sensor, which provides sensor data in a similar way as the complementary environmental data set UDS'. The supplemental environment data set UDS' may be a 4D data set representing a period of time within a future time window. The data set may be considered as a predictive data set representing the environment of the motor vehicle 2.

The evaluation unit 14 of the motor vehicle reads the raw sensor data of the environment sensor 6 and evaluates said data in order to create a 2D or 3D image data set representing the environment of the motor vehicle 4 at a predetermined radius around the motor vehicle 4 and for a predetermined time window. In the present embodiment, the evaluation unit 14 generates a 3D point cloud data set with a radius of 50 meters, for example, around the motor vehicle 4.

The environment 18 is the actual world environment in which the motor vehicle 4 is traveling and includes infrastructure, roads, and other road users.

The environment data transmitter 20 is configured to transmit information about the environment 18, which is evaluated by external sensors (rather than by the motor vehicle 4). This information may be evaluated and transmitted via V2x (vehicle to assessing) communication and evaluated at the recipient (e.g., a pedestrian's smartphone).

Virtual platform 22 resides on cloud computer 8 and is configured to supplement incomplete physical sensor data with additional external sensor data, simulated sensor data, and predicted sensor data. In other words, in the present embodiment, the supplemental data set EDS is generated on the virtual platform 22 and the environmental data set UDS is supplemented with the supplemental data set EDS so that the supplemental environmental data set UDS' can be provided, as will be described in detail below.

The 3D simulation module 24 of the virtual platform 22 provides an environment capable of performing traffic and movement oriented simulations in a 3D manner. The 3D simulation module 24 may include a game engine and/or a traffic simulator for this purpose.

The reconstruction module 26 of the virtual platform 22 is configured to determine a first image data set of the environment of the motor vehicle 4, a second extended image data set of the environment of the motor vehicle 4, and a predicted image data set of the environment of the motor vehicle 4. The determination is based on an x, y, z coordinate system.

The traffic prediction module 28 of the virtual platform 22 is configured to simulate a traffic event in the environment of the motor vehicle 4 in the 3D simulation module 24 so as to extend into the future in a brief time window.

The simulation is based on the provided sensor data. Traffic prediction module 28 may include a physics engine to predict the trajectory of motor vehicle 4 and all virtual road users based on their known locations, speeds, and potential routes (navigation data).

The 3D data set 30 represents a complete 3D image of the static component of the environment of the motor vehicle 4 based on the current position of the motor vehicle 4. For example, based on the GPS coordinates of motor vehicle 4 (e.g., differential GPS with an accuracy of 10 centimeters), all 3D coordinates of streets, buildings, infrastructure, trees, etc. surrounding motor vehicle 4 may be queried from a 3D map database (e.g., open driving, open street maps, Pegasus, etc.).

The positioning module 32 is configured to provide position data indicative of a current position of the motor vehicle 4. A differential GPS of the motor vehicle 4 can be used for this purpose.

The environment data transmitter 34 is configured for exchanging sensor data with road users of the same area of the motor vehicle 4 on the basis of the position data.

The environmental data transmitter 34 may request specific sensor data, such as their location, size, speed and/or direction of movement, from other road users via I2x or V2x or smart phones.

The cloud-side transmitter 36 is configured to exchange data with the motor vehicle 4. Thus, sensor data may be requested from the motor vehicle 4 in order to transmit an improved image of the environment and a prediction of the environment of the motor vehicle 4 to the motor vehicle 4.

The physics engine 38 is a software module, and the physics engine 38 is configured to determine the behavior of all moving elements (which may be considered rigid bodies) according to the laws of physics. For example, the physics engine may be Physx by NVDIA.

The prediction module 40 is configured to detect based on sensor data indicative of current and past conditions in order to provide a 4D data set indicative of the environment of the motor vehicle 4 for a brief period of time in the future.

The method sequence for operating the system 2 in fig. 1 will now be explained with reference to fig. 2 to 5.

The method may be initiated in response to a request by a request signal, for example in response to activation of the driver assistance system 16; or the method is continuously performed without interruption to provide a complementary ambient data set UDS'.

In a first step S100, the control device 10 of the motor vehicle 4 triggers the evaluation unit 14 to read raw sensor data of the environmental sensors 6 and to combine these into an environmental data set UDS. The environment data set UDS may be a 2D or 3D image data set representing the environment of the motor vehicle 4.

Furthermore, in a further substep S110, the control device 10 activates the evaluation unit 14 such that it generates a state data set ZDS indicating the state of the motor vehicle 4. In the present exemplary embodiment, the state data set ZDS comprises position data and/or driving direction data and/or driving speed data of the motor vehicle 4.

Furthermore, in a further sub-step S120, the control device 10 activates the evaluation unit 14 such that it determines the data D to be supplemented in the environment data set UDS, which data to be supplemented are represented as shadow or hollow or blind spot areas and markers in the data environment data set UDS.

In a further step S200, the control device 10 causes the environment data set UDS to be transmitted to the cloud computer 8 via the wireless data link 42.

In a further step S300, the cloud computer 8 generates a supplementary data set EDS for supplementing the ambient data set UDS.

For this purpose, in a first sub-step S310, the cloud computer 8 determines which data are missing by evaluating an environmental data set UDS containing data D to be supplemented, in order to fill in shadow or void or blind spot areas.

Furthermore, in a further substep S320, the cloud computer 8 determines relevant position data and/or driving direction data and/or driving speed data of the motor vehicle 4 by evaluating the state data set ZDS. This makes it possible to determine which data sources can be considered as sources of the data D to be supplemented. In other words, the data source that the motor vehicle 4 has traveled past is discarded.

In a further step S400, the cloud computer 8 supplements the environmental data set UDS with the supplemental data set EDS and thus generates a supplemental environmental data set UDS'.

For this purpose, in a first sub-step S405, in the 3D simulation module 24, the cloud computer 8 reconstructs the 3D scene or a data set DS representing the 3D scene around the motor vehicle 4 on the basis of the environment data set UDS and the state data set ZDS.

In a further substep S410, the motor vehicle 4 is then virtually embedded in the 3D scene or the data set DS representing the scene.

In a further substep S415, the cloud computer 8 reconstructs a first virtual version of the point cloud data set PDS by means of the reconstruction module 26, based on the data set DS around the motor vehicle 4 in the 3D simulation module 24.

In a further sub-step S420, the cloud computer 8 determines the data D to be supplemented, which is represented as a shadow or a void or a blind spot region, by evaluating the first virtual version of the point cloud data set PDS.

In a further sub-step S425, the cloud computer 8 executes a first complementary E1 of a virtual version of the point cloud dataset PDS and adds information, e.g. additional echoes from containment, buildings, trees, etc., to the 3D scene or the dataset DS representing it.

In a further substep S430, it is checked whether the point cloud data set PDS contains a portion EA that still needs to be supplemented.

If the point cloud data set PDS does not comprise the portion EA to be supplemented, in a further sub-step S435 a supplementary environment data set UDS' is generated based on the point cloud data set PDS and the method continues with step S500.

In a further step S500, the supplementary environment data set UDS' is transmitted to the motor vehicle 4 via the wireless data link 42.

However, if the point cloud data set PDS still comprises the portion EA to be supplemented, the method continues with a further substep S440.

In a further sub-step S4340, on the basis of, inter alia, the position data, the cloud computer 8 determines a predetermined travel time of the other road users V at a predetermined distance or away from the motor vehicle 4.

In a further sub-step S445, the cloud computer 8 transmits (e.g. by means of the environmental data transmitter 34) the information query I to the detected road user V (e.g. within a predetermined distance). The other road users V are therefore queried for, in particular, relevant position data and/or driving direction data and/or driving speed data, in other words data of the simulated state data set of the respective detected road user V.

In a further intermediate step S450, the data obtained by the information query I are embedded in a 3D scene or a data set DS representing this scene.

In a further substep S55, the cloud computer 8 performs a further supplementation E2 of the virtual version of the point cloud data set PDS and adds the data of the other road users V obtained by means of the information query I to the 3D scene or to the data set DS representing it. In other words, the acquired data are similar point cloud data sets and/or environmental data sets of the individual road users V.

A further substep S460 may be provided in which the traffic prediction VVH provided by the traffic prediction module 28 is taken into account and/or wirelessly transmitted to the motor vehicle 4.

Then, in a similar manner to the above-described embodiment, in a further sub-step S435, a supplementary ambient data set UDS 'is generated on the basis of the point cloud data set PDS, and in a further step S500, the supplementary ambient data set UDS' is subsequently transmitted to the motor vehicle 4 via the wireless data link 42.

The supplementary surroundings data set UDS' is then associated with the surroundings sensors 6 in the form of sensor data of the virtual sensors and evaluated in order to be able to operate the driver assistance system 16.

The order of steps and/or sub-steps may also be different, unlike embodiments. Furthermore, several steps and/or sub-steps may be performed concurrently or simultaneously. In addition, individual steps may also be omitted.

Computer resources of the motor vehicle 4 can thereby be saved and additional sensor data sources can be used simultaneously, thereby reducing the power requirements of the motor vehicle 4 and improving the quality of the data sensors.

List of reference numerals

2 System

4 Motor vehicle

6 Environment sensor

8 cloud computer

10 control device

12 motor vehicle side transmitter

14 evaluation unit

16 driver assistance system

18 environment

20 environment data transmitter

22 virtual platform

243D simulation module

26 reconstruction module

28 traffic prediction module

303D data set

32 positioning module

34 environment data transmitter

36 cloud side transmitter

38 physics engine

40 prediction module

42 wireless data link

D data to be supplemented

DS data set

E1 supplement

E2 supplement

Part of EA to be supplemented

EDS supplemental data set

I information query

PDS point cloud data set

V road user

VVH traffic prediction

UDS Environment dataset

UDS supplemental Environment dataset

ZDS status data set

S100 step

S200 step

S300 step

S400 step

S500 step

S110 substep

S120 substep

S310 substep

S320 substep

S405 substep

S410 substep

S415 substep

S420 substep

S425 substep

S430 substep

S435 substep

S440 substep

S445 substep

S450 substep

S455 substep