阅读说明：本技术 雷达、lidar和超声测量数据的合成生成 (Synthetic generation of radar, LIDAR and ultrasound measurement data ) 是由 T·宾泽尔 A·霍列娃 J·哈施 于 2020-11-03 设计创作，主要内容包括：雷达、LIDAR和超声测量数据的合成生成。一种用于生成与由第一物理测量模式捕获的实际测量数据不能区分开的合成测量数据的方法,其中,该第一物理测量模式基于向对象发射询问波并且以允许确定询问射束的发射与反射波的到达之间的飞行时间的方式记录来自对象的反射波,该方法包括以下步骤：获得第一潜在空间中的合成测量数据的第一压缩表示,其中,该第一潜在空间与第一解码器相关联,该第一解码器将第一潜在空间的每个元素映射到与第一物理测量模式的实际测量数据的记录不能区分开的合成测量数据的记录；以及将第一解码器应用于第一压缩表示,以便于获得所寻求的合成测量数据。用于训练第一编码器和解码器以及用于训练域变换的方法。(Synthetic generation of radar, LIDAR and ultrasound measurement data. A method for generating synthetic measurement data indistinguishable from actual measurement data captured by a first physical measurement mode based on transmitting an interrogation wave to an object and recording reflected waves from the object in a manner that allows time of flight between the transmission of the interrogation beam and the arrival of the reflected waves to be determined, the method comprising the steps of: obtaining a first compressed representation of the synthetic measurement data in a first potential space, wherein the first potential space is associated with a first decoder that maps each element of the first potential space to a record of synthetic measurement data indistinguishable from a record of actual measurement data of a first physical measurement pattern; and applying a first decoder to the first compressed representation in order to obtain the sought synthetic measurement data. Methods for training a first encoder and decoder and for training a domain transform.)

1. A method (100) for generating synthetic measurement data (3) indistinguishable from actual measurement data (1) captured by a first physical measurement mode based on transmitting an interrogation wave to an object and recording reflected waves from the object in a manner that allows determining a time of flight between transmission of the interrogation beam and arrival of the reflected waves, the method comprising the steps of:

● obtaining (110) a first compressed representation (3) of synthetic measurement data (3) in a first potential space (1 b), wherein the first potential space (1 b) is associated with a first decoder (1 c), said first decoder (1 c) being trained to map each element of the first potential space (1 b) to a record of synthetic measurement data indistinguishable from a record of actual measurement data (1) of a first physical measurement pattern; and

● applies (120) the first decoder (1 c) to the first compressed representation (3) in order to obtain the sought synthetic measurement data (3).

2. The method (100) according to claim 1, wherein obtaining (110) a first compressed representation (3) comprises in particular:

●, extracting (111) samples (5) from an input space (5 a) of a prior transformation (5), said prior transformation (5) being trained to map each element of the input space (5 a) to an element of a first potential space (1 b); and

● applies (112) an a priori transform (5) to the samples (5) in order to obtain the sought first compressed representation (3).

3. The method (100) according to claim 2, wherein the a priori transformation (5) particularly comprises at least one trained autoregressive neural network.

4. The method (100) according to any one of claims 2 to 3, wherein the a priori transform (5) comprises a plurality of portions (51-53), such that different portions (51-53) of the a priori transform (5) map elements of their respective input spaces (51 a-53 a) to respective outputs (51 b-53 b), which are then superimposed to form the first compressed representation (3).

5. The method (100) according to claim 1, wherein obtaining (110) a first compressed representation (3) comprises in particular:

● applying (113) the second trained encoder (2 a) to a record of actual measurement data (2) of a second physical measurement pattern, the second physical measurement pattern being different from the first physical measurement pattern, in order to obtain a second compressed representation (2 ″) of the actual measurement data (2) in a second potential space (2 b), wherein the second potential space (2 b) is associated with a second decoder (2 c), the second decoder (2 c) being trained to map each element of the second potential space (2 c) to a record of synthetic measurement data (2') indistinguishable from the record of actual measurement data (2) of the second physical measurement pattern; and

● applies (114) a domain transform (6) to the second compressed representation (2), the domain transform being trained to map each element of the second potential space (2 b) to an element of the first potential space (1 b) in order to obtain the sought first compressed representation (3).

6. The method (100) according to claim 5, wherein the second physical measurement mode comprises in particular: a spatially resolved distribution of the intensity and/or wavelength of the light wave incident on the sensor is recorded.

7. The method (100) according to any one of claims 1 to 6, wherein the first compressed representation (1) comprises a vector or tensor of discrete variables, wherein the number of these variables is smaller than the number of variables in the recording of the measurement data (1, 3) of the first physical measurement mode.

8. The method (100) according to any one of claims 1 to 7, wherein obtaining (110) the first compressed representation (3) in particular comprises: -picking a first potential space (1 b) from which said first compressed representation (3) is obtained, such that the first decoder (1 c) reconciles the synthetic measurement data (3) to which the elements of this first potential space (1 b) are mapped with at least one predetermined condition (7).

9. The method (100) according to claim 8, wherein the predetermined condition (7) comprises in particular: interaction of the interrogating wave with one or more particular objects, and/or one or more environmental conditions affecting propagation of the interrogating wave and/or reflected wave.

10. The method (100) according to any one of claims 1 to 9, further comprising: training (130) at least one machine learning module (54) using the generated synthetic measurement data (3), the machine learning module (54) mapping actual measurement data (1) captured from the vehicle (50) to at least one classification and/or regression value, wherein the classification and/or regression value is related to operating the vehicle (50) in road traffic in an at least partially automated manner.

11. The method of claim 10, further comprising:

● obtaining (140) actual measurement data (1) from the vehicle (50) using a first physical measurement mode;

● processing (150) the acquired actual measurement data (1) by the trained machine learning module (54) to obtain at least one classification and/or regression value (8);

● calculating (160) at least one actuation signal (9) for at least one system (55) of the vehicle (50) from the classification and/or regression values (8); and

● actuates (170) the system (55) with the actuation signal (9).

12. The method (100) according to any one of claims 1 to 11, wherein the interrogation wave of the first physical measurement mode is a radar wave, a LIDAR wave or an ultrasound wave.

13. The method (100) according to claim 12, wherein the measurement data (1) of the first physical measurement mode comprises:

● to the point on the object where the reflected wave is being transmitted; and/or

● point clouds of the locations on the object.

14. A method (200) of training a first encoder (1 a) and decoder (1 c) for use in the method (100) of any one of claims 1 to 13, comprising the steps of:

● obtaining (210) a record of a set of actual measurement data (1) by a first physical measurement mode;

● mapping (220) each record of the actual measurement data (1) to a first compressed representation (3) by means of a first trainable encoder (1 a);

● mapping (230) the first compressed representation (3) to a record of synthetic measurement data (3) of the first physical measurement pattern by means of the first trainable decoder (1 c); and

● optimizing (240) the parameters (1 a, 1 c) characterizing the behaviour of the first encoder (1 a) and the decoder (1 c), with the aim of minimizing the differences between the recording of the synthetic measurement data (3) and the corresponding recording of the actual measurement data (1),

wherein the content of the first and second substances,

● A first physical measurement mode is based on transmitting an interrogation wave to the object and recording reflected waves from the object in a manner that allows the time of flight between the transmission of the interrogation beam and the arrival of the reflected waves to be determined, an

● the interrogation wave is a radar wave, a LIDAR wave, or an ultrasonic wave.

15. A method (300) of training a domain transform (6) for use in the method (100) of any one of claims 5 to 6, comprising the steps of:

● obtaining (310) a record of a set of actual measurement data (2) by a second physical measurement mode;

● mapping (320) each record of the actual measurement data (2) to a second compressed representation (2 x) by means of a second trained encoder (2 a);

● mapping (330) the second compressed representation (2) to the first compressed representation (3) by means of the trainable domain transformation (6);

● mapping (340) the first compressed representation (3 x) to a record of synthetic measurement data (3) by means of the first trained decoder (1 c); and

● optimizes (350) parameters (6) characterizing the behavior of the domain transform (6) with the aim of making the record of the synthetic measurement data (3) indistinguishable from the record resulting from processing the actual measurement data (1) of the first physical measurement mode into a compressed representation (3) using the first trained encoder (1 a) and passing the compressed representation (3) to the first trained decoder (1 c).

16. A computer program comprising machine-readable instructions which, when executed by one or more computers, cause the one or more computers to carry out the method (100, 200, 300) according to any one of claims 1 to 15.

17. A non-transitory machine-readable storage medium and/or download product having one or more of the following:

● the computer program of claim 16;

● synthetic measurement data (3) generated by the method (100) according to any one of claims 1 to 13;

● characterizing the behaviour of the first encoder (1 a) and decoder (1 c) and the parameters (1 a, 1 c) generated by the method (200) according to claim 14; and

● characterizing the behavior of the domain transform (6) and the parameters (6) generated by the method (300) according to claim 15.

18. A computer provided with a computer program according to claim 16 and/or having a machine-readable storage medium according to claim 17 and/or a download product.

Technical Field

The invention relates to the synthetic generation of measurement data, in particular for data corresponding to radar, LIDAR (laser radar), ultrasound and similar physical measurement modes.

Background

In order to guide a vehicle through road traffic in an at least partially automated manner, it is necessary to capture physical measurement data from the surroundings of the vehicle and to evaluate this data for other traffic participants, lane boundaries or any other kind of object whose appearance may require a change in the trajectory of the vehicle.

Regardless of the lighting conditions, the object may be captured by radar. Moreover, the radar data will immediately derive the distance to the object and the speed of the object. This is information that is crucial for assessing whether the vehicle is likely to collide with the detected object.

When a machine learning module is to be trained to recognize objects based on radar measurements, the training data that needs to be trained is a scarce resource. As detailed in german patent DE 102018204494B 3, when training data needs to be labeled by humans to perform supervised learning, the task is more difficult than for images, since identifying objects from radar signals is far less intuitive. Moreover, object recognition based on radar data tends to require more training data than object recognition based on optical images, since there are many factors that affect the propagation of radar waves. Thus, the DE 102018204494B 3 patent suggests the use of a generation countermeasure network GAN to generate synthetic radar data.

Disclosure of Invention

The inventors have developed a method for generating synthetic measurement data indistinguishable from actual measurement data captured by a first physical imaging modality. This physical imaging mode is based on transmitting an interrogation wave towards the object and recording reflected waves from the object in a manner that allows the time of flight between the transmission of the interrogation beam and the arrival of the reflected waves to be determined. In the form of a directional beam, an interrogation wave may be transmitted and a reflected wave may be received.

In particular, the interrogation wave may be a radar wave, a LIDAR wave or an ultrasound wave. In particular, the measurement data of the first physical measurement mode may comprise

● to the point on the object where the reflected wave is being transmitted; and/or

● point clouds of the locations on the object.

Common to these measurement modes is that raw data is more difficult for humans to interpret than image data. Therefore, when a machine learning model is trained to classify an object indicated by measurement data or to obtain a regression such as the speed of the object, it may be more expensive and time consuming to label the record of training data with a "ground truth" related to the task at hand. The possibility of obtaining synthetic measurement data allows to make more training data available for the training of the machine learning module without having to pay too much manpower for labeling the data.

The measurement data may take any form suitable for the intended use. For example, the measurement data may include time series data or a transformation of the time series data into frequency space, such as a fast fourier transform.

The method begins by obtaining a first compressed representation of synthetic measurement data in a first potential space. The first potential space is associated with a first decoder trained to map each element of the first potential space to a record of synthetic measurement data indistinguishable from a record of actual measurement data of the first physical measurement pattern. For example, the concatenation of encoder and decoder (tandem) may be trained such that the record of synthetic measurement data preferably corresponds to the original record of actual measurement data when the record of actual measurement data is transformed by the encoder into a first compressed representation and then back into a record of synthetic measurement data. Even with such a tandem training, only the first decoder in its training state is required to carry out the method.

A trained first decoder is applied to the first compressed representation. This produces the composite measurement data sought.

The inventors have found that in this way the rather complex task of obtaining synthetic measurement data can be simplified to a much simpler task of finding a suitable compressed representation. In particular, if the compression is a lossy compression, the compressed representation may be 100 times smaller or less than the record of measurement data in terms of the amount of data involved. It is much easier to find things in a space that is 100 times less dimensional. The raw data generated by radar, LIDAR and ultrasonic sensors is sparse, i.e. contains much less information than the corresponding signals can represent. This allows lossy compression of the data without losing any critical information about the sampling scenario.

This effect is particularly pronounced if the first compressed representation comprises vectors or tensors of discrete variables, and the number of these variables is smaller than the number of variables in the recording of the measurement data of the first physical measurement mode. For example, the first compressed representation may be a vector quantized representation, i.e. a vector or tensor having quantized components. This reduces the space in which the first compressed representation is further sought: since there are a limited number of dimensions and a limited number of possible discrete values along each dimension, there is a limited number of possible compressed representations.

The first compressed representation may be sought in the reduced space using any suitable technique. For example, parametric optimization techniques can be used to optimize variables in the compressed representation in order to find a representation that maps to a suitable record of the synthetic measurement data (i.e., a record that is indistinguishable from the actual measurement data). In the case of vector quantization, even a brute force search of the first potential space may be feasible if the candidate compressed representation can be tested fast enough whether it maps to a suitable record of synthetic measurement data.

In the following, two exemplary methods of obtaining the first compressed representation are disclosed. These exemplary methods also take into account that the first potential space is typically a subspace of the complete vector space spanned by all variables taken as input by the decoder. For example, if an encoder-decoder tandem is trained and the compressed representation produced by the encoder is a vector with 100 components, not all vectors from the 100-dimensional vector space will be decoded into meaningful results. Additional machine learning may be used to train the ability to find vectors that do map to the appropriate records of the composite measurement data.

In a first exemplary embodiment, samples are extracted from an input space of an a priori transform trained to map each element of the input space to an element of a first potential space. An a priori transform is then applied to the samples and this produces the first compressed representation sought. This first compressed representation then represents the appropriate record to be mapped to the synthetic measurement data. In this way, the overall task of obtaining composite measurement data is split into two tasks that can be performed in turn: an encoder-decoder series is trained to form a first potential space, and then an a priori transform is trained to convert samples (e.g., random samples) from an input space to samples that are members of the first potential space.

For example, the a priori transformation may specifically comprise at least one trained autoregressive neural network. For example, the neural network may be a convolutional neural network. One example of an a priori transformation that can convert arbitrary input data into members of a potential space created by an encoder-decoder tandem is known in the art as "PixelCNN".

Preferably, the a priori transform may comprise a plurality of portions, such that different portions of the a priori transform map elements of their respective input spaces to respective outputs, which are then superimposed to form the first compressed representation. For example, different parts of the a priori transformation may map their respective inputs to different parts of the first compressed representation, and all parts together form the complete first compressed representation. This is particularly advantageous in case the first compressed representation is organized into a hierarchical structure having a plurality of levels. Different portions of the a priori transformation may then be trained to produce different levels of the first compressed representation.

The advantage of splitting the a priori transform in this manner is twofold.

First, there is greater flexibility in adding conditions to the synthetic measurement data sought. For example, the task at hand may not just find any synthetic radar images, but rather images that would indicate the presence of certain objects. This presence of certain objects may form a "class label" in the classification task. In an example where the first compressed representation includes three levels (called the highest, medium, and lowest levels), a first autoregressive neural network (e.g., PixelCNN) may be trained to find each level of the compressed representation. The highest level of acquisition may depend on the category label; the lower level of acquisition may depend on the category label and the results of the previous level.

Second, since the various parts of the a priori transform can be trained separately, the training can be parallelized. In this way, all available computing power and memory on the hardware accelerator may be utilized.

In a second exemplary embodiment, the method is specifically configured for domain transfer of measurement data. That is, starting from the actual measurement data of the second physical measurement mode, synthetic measurement data of the first physical measurement mode representing a scene with substantially similar contents is sought.

For example, the second physical measurement mode may specifically include: a spatially resolved distribution of the intensity and/or wavelength of the light wave incident on the sensor is recorded. Such sensors produce images that can be easily interpreted by humans. It is therefore very common practice to obtain labels representing the types of objects contained in the images by distributing the tasks to a large number of human workforce, which will mark the objects they recognize in the images. There are also many existing collections of such label images. If such a tag image domain is transferred to a composite record of radar data, it is known from the outset which objects the radar data will represent. In other words, any "ground truth" tags attached to the image can be reused for the radar data. This is much easier than manually marking radar data from scratch. Such manual tagging of radar data requires more expertise and more time than manual tagging of images.

To accomplish the domain transfer, a second trained encoder is applied to the recording of actual measurement data for a second physical measurement mode, which is different from the first physical measurement mode. For example, the record may include an image.

A second trained encoder is applied to produce a second compressed representation of second actual measurement data (e.g., an image) in a second potential space. Similar to the first potential space, the second potential space is associated with a second decoder trained to map each element of the second potential space to a record of synthetic measurement data of the second measurement mode indistinguishable from a record of actual measurement data of the second physical measurement mode. For example, the second encoder and second decoder may be trained in an encoder-decoder tandem manner such that when an input record (e.g., an image) of actual measurement data of the second mode is encoded by the encoder into a compressed representation and subsequently decoded by the decoder, the input record is best reproduced. Even if the second encoder and the second decoder are trained in series in this way, only the second encoder needs to be in its training state during the method.

The domain transform is applied to the second compressed representation. The domain transform is trained to map each element of the second potential space to an element of the first potential space. In this way, the sought first compressed representation is obtained. A first decoder may then be applied to this first compressed representation to obtain the final result, i.e. the sought synthetic measurement data of the first physical measurement mode.

Similar to the first embodiment, the task of domain transfer is split into the generation of a second compressed representation on the one hand and the actual transfer of this second compressed representation to the first potential space on the other hand. The training for the two tasks may be carried out again in sequence and is therefore easier to accomplish than one single training of the overall mapping which directly leads to the first compressed representation in the first potential space from the recording of the actual measurement data of the second mode. Here, the real-world analogy is that jumping 1 m high from the ground to the first step, and then jumping 1 m high from there to the next step is much easier than jumping 2 m high at a time.

In a further particularly advantageous embodiment, whether the task is to obtain the synthetic measurement data "from scratch" (e.g. based on arbitrary samples randomly extracted from the input space) or by domain transfer from another measurement mode, the first potential space from which the first compressed representation is obtained may be chosen such that the synthetic measurement data to which the first decoder maps elements of this first potential space coincides with the at least one predetermined condition. As discussed previously, the predetermined condition may include some sort of class label so that synthetic measurement data belonging to a particular class of classification may be obtained. For example, radar data showing two vehicles and a stop sign during a collision may be particularly desirable.

Preferably, the predetermined conditions specifically include: interaction of the interrogating wave with one or more particular objects, and/or one or more environmental conditions affecting propagation of the interrogating wave and/or reflected wave. For example, as the microwave radiation used for radar imaging is partially absorbed by water, the radar data for the same scene may change when heavy rain comes in. An object detection system for a vehicle should operate reliably in a variety of environmental conditions and therefore requires training data that has some variability with respect to these environmental conditions.

By appropriately setting the predetermined conditions, the method can be used to obtain training data representing various conditions and combinations, even if not all such combinations have become part of the training of any of the used encoders, decoders, a priori transforms and domain transforms. For example, such training may be based on radar data for various types of vehicles and various types of weather conditions, but may not have data for lanborkini in heavy snowfall, as the wisdom owner of such expensive cars does not risk an accident under adverse winter driving conditions. With the above method, the radar data may be synthetically generated, thereby expanding the pool of data that may be used to train a machine learning module for object detection.

Thus, in a further particularly advantageous embodiment, the method may further comprise: the at least one machine learning module is trained using the generated synthetic measurement data of the first physical measurement mode. The machine learning module is to map actual measurement data captured from the vehicle to at least one classification and/or regression value. The classification and/or regression values relate to operating the vehicle in road traffic in an at least partially automated manner. In particular, the generated synthetic measurement data may be used to enhance an already existing set of actual measurement data of the first physical measurement mode such that the final data set used for training has a desired variability with respect to different situations and conditions.

As previously discussed, the category of the classification may be related to the type of object. In particular, measurement data acquired from a vehicle may be "semantically segmented" into contributions from different objects. For example, regression values may include: the speed and direction of the object, the coefficient of friction between the tires and the road, or the maximum range ahead of the vehicle in the direction of travel that can be surveyed by the vehicle's sensors under the current conditions.

After training it in this way, the machine learning module can be put into use in a vehicle. Thus, in a further particularly advantageous embodiment, the method further comprises:

● obtaining actual measurement data from the vehicle using a first physical measurement mode;

● processing the acquired actual measurement data by a trained machine learning module to obtain at least one classification and/or regression value;

● calculating at least one actuation signal for at least one system of the vehicle from the classification and/or regression values; and

● actuate the system with the actuation signal.

For example, upon determining that a currently contemplated vehicle's trajectory in space and time intersects the trajectory of another vehicle in transit, the steering system and/or braking system may be actuated to slow the vehicle to a stop, or to walk on a path around another vehicle, before reaching the other vehicle. In this and further safety-critical applications, the possibility of generating synthetic measurement data allows to enlarge the training data used for training the machine learning module, thereby improving the variability of the training data. This improves the result of the training and thus also improves the likelihood that the machine learning module will cause the vehicle to perform the correct action given the particular traffic situation.

As discussed previously, the first decoder in its trained state may be obtained based on unmarked actual measurement data using training in a variant auto-encoder (variational auto-encoder) style. The invention therefore also relates to a method for training a first encoder and a decoder. The method comprises the following steps:

● obtaining a record of a set of actual measurement data by a first physical measurement mode;

● mapping each record of actual measurement data to a first compressed representation by means of a first trainable encoder;

● mapping the first compressed representation to a record of synthetic measurement data of the first physical measurement mode by means of the first trainable decoder; and

● optimizes parameters characterizing the behavior of the first encoder and decoder, with the goal of minimizing the difference between the record of synthetic measurement data and the corresponding record of actual measurement data.

In this context, as discussed previously, this first physical measurement mode is based on transmitting an interrogation wave to the object and recording reflected waves from the object in a manner that allows the time of flight between the transmission of the interrogation beam and the arrival of the reflected waves to be determined. The interrogation wave is a radar wave, a LIDAR wave, or an ultrasonic wave.

As discussed previously, the training of the first decoder is independent of the training of any other means used to obtain the first compressed representation, such as an a priori transform or a domain transform. This means that if the a priori transformation is to be changed to a transformation that maps between the new desired input space and the first potential space, the training of the first decoder remains valid and does not have to be repeated. Likewise, if it is desired to effect a domain transfer from the new physical measurement mode to the first physical measurement mode, a new domain transform will have to be trained, but no change to the first decoder will be required.

The present invention also provides a method for training a domain transform that may be used to map a second compressed representation of actual measurement data of a second physical measurement mode to a first potential space.

During the method, a record of a set of actual measurement data is obtained by a second physical measurement mode. Each record of this actual measurement data is mapped to a second compressed representation by means of a second trained encoder, which may be trained in tandem with a second decoder in the fashion of a variational auto-encoder, as discussed previously.

The second compressed representation is mapped to the first compressed representation by a trainable domain transform. The first compressed representation is mapped to a record of synthetic measurement data by means of a first trained decoder. Optimizing the parameters characterizing the behavior of the domain transform aims at making the record of the synthetic measurement data indistinguishable from a record resulting from processing the actual measurement data of the first physical measurement mode into a compressed representation using the first trained encoder and passing the compressed representation to the first trained decoder.

In other words, the optimization criterion for optimizing the parameters characterizing the behavior of the domain transform measures the degree to which the finally obtained recording synthetic measurement data is "mixed" among the recordings that have been generated from the known members of the first underlying space, that is to say the first compressed representation has been generated from the actual measurement data of the first physical measurement mode by means of the first encoder corresponding to the first decoder.

In any of the training methods, the parameters may include, for example: weights at which inputs to a neuron or other processing unit in a neural network are aggregated to form an activation of the neuron or other processing unit. The optimization of the parameters may be performed according to any suitable method. For example, a gradient descent method may be used.

All of the methods described above may be at least partially computer-implemented. The invention therefore also relates to a computer program having machine-readable instructions which, when executed by one or more computers, cause the one or more computers to carry out at least one of the above-mentioned methods. In this respect, the term "computer" is intended to include electronic control units for vehicles or vehicle subsystems, as well as other embedded systems for controlling technical equipment on the basis of programmable instructions.

The computer program may be embodied in a non-transitory machine-readable storage medium and/or in a downloaded product. The download product is a digital deliverable product that can be traded and purchased online so that it can be immediately delivered to the computer without having to transport the non-transitory storage medium.

Alternatively or in combination, the storage medium and/or the download product may contain synthetic measurement data generated by the method as described above. As discussed above, anyone with this composite measurement data can immediately begin to enhance the training of the machine learning module.

Alternatively or in combination, the storage medium and/or the download product may contain parameters characterizing the behavior of the first encoder and decoder, as well as parameters resulting from a training method for such an encoder and decoder. Anyone with these parameters can immediately start using the first encoder and the first decoder without training them.

Alternatively or in combination, the storage medium and/or the download product may contain parameters characterizing the behavior of the domain transformations, as well as parameters resulting from the training method for such domain transformations. Anyone with these parameters can immediately start using the domain transform without training it.

The invention also relates to a computer provided with a computer program and/or with a machine-readable storage medium and/or with a download product.

Further advantageous embodiments will now be described in detail using the figures, without intending to limit the scope of the invention.

Drawings

The figures show:

FIG. 1: an exemplary embodiment of a method 100 for generating synthetic measurement data 3;

FIG. 2: a schematic overview of the involved spaces and transformations used in the process of method 100;

FIG. 3: an exemplary embodiment of a method 200 for training a first encoder 1a and a first decoder 1 c;

FIG. 4: an exemplary embodiment of a method 300 for training a domain transform 6.

Detailed Description

Fig. 1 is a flow chart of an exemplary embodiment of a method 100. In step 110, a first compressed representation of the synthetic measurement data in the first potential space 1b is obtained 3 x. As it is illustrated in fig. 2 and 3, this first potential space 1b is associated with a first decoder 1c that can be trained in series with the first encoder 1 a. In step 120, the first decoder 1c is applied to the first compressed representation 3 × so that the sought synthetic measurement data is obtained.

Within box 110, two exemplary ways of obtaining the first compressed representation 3 are illustrated.

Samples 5 may be extracted from the input space 5a of the a priori transform 5, according to block 111. The a priori transformation 5 is trained to map each element of the input space 5a to an element of the first potential space 1 b. Thus, when the mapping is carried out according to block 112, the first compression representation sought is 3 x.

In the example shown in fig. 1, the a priori transform 5 comprises a plurality of portions 51-53. Each of these multiple portions 51-53 has its own input space 51a-53a and outputs contributions 51b-53 b. Contributions 51b-53b are aggregated to form a compressed representation 3.

According to block 113, a second trained encoder 2a may be applied to the recording of the actual measurement data 2 of the second physical measurement mode. This produces a second compressed representation 2 of the actual physical measurement data. According to block 114, the domain transform 6 may be applied to the compressed representation 2 to obtain a first compressed representation 3 in the first potential space 1 b.

Additionally, the use to which the obtained synthetic measurement data 3 can be put is illustrated in fig. 1. In step 130, these synthetic measurement data 3 may be used to enhance the training of the machine learning module 54 in order to obtain its training state 54. The trained machine learning module 54 may in turn be used in step 150 to process the actual physical measurement data 1 that has been captured from the vehicle 50 in step 140.

The processing in step 150 generates at least one classification and/or regression value 8 related to the operation of the vehicle 50 in traffic. From the classification and/or regression values 8, in step 160 at least one actuation signal 9 for the system 55 of the vehicle 50 is calculated. In step 170, the system 55 is actuated using the actuation signal 9.

Fig. 2 illustrates the transformations and spaces involved. From the actual measurement data 1 of the first physical measurement mode, the first encoder 1a generates a compressed representation 3 residing in the potential space 1 b. The first decoder 1b is trained to map each compressed representation 3 x to a record of synthetic measurement data 3 of the first physical measurement pattern. I.e. the compressed representation 3 is also a representation of the record of the synthetic measurement data 3. The first encoder 1a and the first decoder 1b can be trained in tandem with the optimization goal that the synthetic measurement data 3 finally obtained from a given recording of the actual measurement data 1 should best match the original measurement data 1. This optimization target indicated by a dotted line makes the synthetic measurement data 3 indistinguishable from the actual measurement data 1.

Likewise, for a second physical measurement pattern, different from the first physical measurement pattern, there is a second encoder 2a, which second encoder 2a maps the actual measurement data 2 of this second pattern to a compressed representation 2 x residing in a second potential space 2 b. The second decoder 2c maps the compressed representation 2 x to a record of the synthetic measurement data 2' of the second measurement pattern. The second encoder 2a and the second decoder 2b can be trained in series with the optimization goal that the resulting measurement data 2' finally obtained from a given recording of the actual measurement data 2 should best match the original actual measurement data 2. This optimization objective, indicated with a dashed line, makes the synthetic measurement data 2' indistinguishable from the actual measurement data 2.

One way of obtaining the synthetic measurement data 3 of the first physical measurement mode is to extract samples 5 from the input space 5a of the prior transformation 5 and then apply the prior transformation 5 to obtain a compressed representation 3, which is then converted by the first decoder 1c into the synthetic measurement data 3 sought.

Another way of obtaining the synthetic measurement data 3 is to perform a domain transfer from the second physical measurement mode. From the actual measurement data 2 of this second pattern, the second encoder 2a generates a compressed representation 2. The trained domain transform 6 transforms the compressed representation 2 from the second potential space 2b into a compressed representation 3 in the first potential space 1b, which compressed representation 3 can again be converted by the first decoder 1c into the sought synthetic measurement data 3.

The first potential space 1b may be specifically chosen such that the synthetic measurement data 3 obtained by the first decoder 1c from its members satisfies the desired condition 7.

Fig. 3 is a flow chart of an exemplary embodiment of a method 200 for tandem training the first encoder 1a and the first decoder 1 c. In step 210, actual measurement data 1 of a first physical measurement mode is obtained. In step 220, the measurement data 1 is converted into a compressed representation 3. In step 230, synthetic measurement data 3 is obtained from the compressed representation 3. In step 240, this synthetic measured data 3 is compared with the original actual measured data 1 and the parameters 1a, 1c characterizing the behavior of the first encoder 1a and the first decoder 1c are optimized for the best match between the synthetic data 3 and the original data 1.

Fig. 4 is a flow diagram of an exemplary embodiment of a method 300 for training domain transform 6. In step 310, actual measurement data 2 of the second physical measurement mode is obtained. In step 320, this data 2 is mapped into a second compressed representation by the second encoder 2a already in its training state. In step 330, the compressed representation 2 residing in the second potential space 2b is transformed into a compressed representation 3 residing in the first potential space 1b by the domain transform 6 to be trained. In step 340, the compressed representation 3 is mapped to the record of synthetic measurement data 3 by the first decoder 1c already in its training state. In step 350, this synthetic gauging data 3 is compared with the result obtained when the actual physical gauging data 1 of the first physical gauging pattern was first transformed into a compressed representation 3 x by the first decoder 1a already in its training state and then transformed back into gauging data 3. The parameters 6 characterizing the behavior of the domain transformation 6 are optimized such that the best match between the synthetic measured data 3 generated from the actual measured data 2 of the second physical measuring modality via the domain transformation 6 and the synthetic measured data 3 generated from the actual measured data 1 of the first physical measuring modality is obtained.