阅读说明：本技术 一种目标关联方法、装置、电子设备及存储介质 (Target association method and device, electronic equipment and storage medium ) 是由 付仁涛 王相玲 吕颖 王祎男 关瀛洲 魏源伯 刘汉旭 于 2021-08-27 设计创作，主要内容包括：本申请实施例公开了一种目标关联方法、装置、电子设备及存储介质。该方法包括：将第一传感器识别到的至少一个物体作为参考目标,并获取至少一个参考目标中的各个参考目标的位置信息；将第二传感器识别到的至少一个物体作为目标,并获取至少一个目标中的各个目标的位置信息；为至少一个参考目标中的各个参考目标设置对应的关联门限；从至少一个目标中剔除掉不在任意一个关联门限内的目标,将至少一个目标中剩余的各个目标作为待关联目标；将至少一个参考目标中的各个参考目标与至少一个待关联目标中的各个待关联目标进行关联,得到关联关系表。本申请的技术方案,可以提高目标关联的准确率,并能够通过降低目标关联时的计算量满足实时性要求。(The embodiment of the application discloses a target association method, a target association device, electronic equipment and a storage medium. The method comprises the following steps: taking at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target; setting a corresponding association threshold for each reference target in at least one reference target; removing targets which are not in any association threshold from at least one target, and taking each residual target in the at least one target as a target to be associated; and associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table. According to the technical scheme, the accuracy of target association can be improved, and the real-time requirement can be met by reducing the calculated amount during target association.)

1. A method for target association, the method comprising:

respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target;

setting a corresponding association threshold for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target;

setting a corresponding association threshold and position information of each target in the at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated;

associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table; the association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

2. The method of claim 1, wherein the association threshold is a variable rectangle threshold; the position information includes at least a lateral distance and a longitudinal distance;

the setting a corresponding association threshold for each reference target of the at least one reference target according to the position information of each reference target of the at least one reference target includes:

and determining the length value and the width value of the variable rectangular threshold corresponding to each target in the at least one reference target according to the longitudinal distance of each reference target in the at least one reference target.

3. The method of claim 1, further comprising:

acquiring speed information of each reference target in the at least one reference target and speed information of each target to be associated in the at least one target to be associated;

calculating the speed difference between the reference target and the target to be associated in each association pair in the association relation table according to the speed information of each reference target in the at least one reference target and the speed information of each target to be associated in the at least one target to be associated;

and if the speed difference is not within the range of the preset threshold value, removing the association pair from the association relation table.

4. The method according to claim 1, wherein associating each of the at least one reference target with each of the at least one object to be associated to obtain an association relationship table comprises:

establishing a correlation matrix between the at least one reference target and the at least one target to be correlated according to the position information of each reference target in the at least one reference target and the position information of each target to be correlated in the at least one target to be correlated;

and transforming the incidence matrix according to a preset transformation rule to obtain a transformed incidence matrix, and calculating the transformed incidence matrix to obtain an incidence relation table.

5. The method according to claim 4, further comprising, after establishing a correlation matrix between the at least one reference object and the at least one object to be correlated:

and if the number of the at least one reference target is inconsistent with the number of the at least one target to be associated, filling the association matrix to enable the association matrix to be a square matrix.

6. The method of claim 1, further comprising, after obtaining the location information of each of the at least one reference target and obtaining the location information of each of the at least one target:

and if the position information of the at least one reference target and the position information of the at least one target are not in the same coordinate system, respectively converting the position information of the at least one reference target and the position information of the at least one target into position information in the same coordinate system.

7. The method of claim 1, further comprising, after obtaining the location information of each of the at least one reference target and obtaining the location information of each of the at least one target:

and respectively correcting the position information of each reference target in the at least one reference target and the position information of each target in the at least one target under the same coordinate system through a pre-trained model according to the recognition time of the at least one reference target and the recognition time of the at least one target, so as to obtain the corrected position information of the at least one reference target and the corrected position information of the at least one target.

8. An apparatus for target association, the apparatus comprising:

the information acquisition module is used for respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target and acquiring position information of each reference target in the at least one reference target; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target;

a threshold setting module, configured to set a corresponding association threshold for each reference target of the at least one reference target according to the position information of each reference target of the at least one reference target;

a target determining module, configured to set a corresponding association threshold and position information of each target of the at least one target based on each reference target, remove targets that are not within any association threshold from the at least one target, and use each remaining target of the at least one target as a target to be associated;

the target association module is used for associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table; the association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

9. An electronic device, characterized in that the electronic device comprises:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the target association method as recited in any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the object association method according to any one of claims 1 to 7.

Technical Field

The embodiment of the application relates to a multi-sensor association technology, and in particular relates to a target association method, a target association device, an electronic device and a storage medium.

Background

In the field of automatic driving technology, intelligent driving is divided into five levels, and as the intelligent driving level is increased, the types and the number of sensors arranged in a vehicle are increased, so that the multi-sensor correlation technology is a necessary trend.

In the multi-sensor correlation technique, an important issue is how to determine that a plurality of targets from different sensors are the same target, and to correlate the same target. In the prior art, common target association methods include a nearest neighbor algorithm, a joint probability data association algorithm and a multi-hypothesis tracking algorithm. The nearest neighbor algorithm is high in real-time performance, but can be only used in an environment with sparse targets, and the problem of mistaken association or target loss is easy to occur in an environment with dense targets; the joint probability data association algorithm and the multi-hypothesis tracking algorithm can be used in an environment with dense targets, the association accuracy is high, the calculation amount is large, and the real-time requirement is difficult to meet. Therefore, it is necessary to design a target association method, which can improve the accuracy of target association, reduce the amount of calculation, and meet the real-time requirement.

Disclosure of Invention

The embodiment of the application provides a target association method, a target association device, an electronic device and a storage medium, which can improve the accuracy of target association and meet the real-time requirement by reducing the calculated amount during target association.

In a first aspect, an embodiment of the present application provides a target association method, where the method includes:

respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target;

setting a corresponding association threshold for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target;

setting a corresponding association threshold and position information of each target in the at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated;

associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table; the association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

In a second aspect, an embodiment of the present application provides an object associating apparatus, including:

the information acquisition module is used for respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target and acquiring position information of each reference target in the at least one reference target; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target;

a threshold setting module, configured to set a corresponding association threshold for each reference target of the at least one reference target according to the position information of each reference target of the at least one reference target;

a target determining module, configured to set a corresponding association threshold and position information of each target of the at least one target based on each reference target, remove targets that are not within any association threshold from the at least one target, and use each remaining target of the at least one target as a target to be associated;

the target association module is used for associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table; the association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

In a third aspect, an embodiment of the present application provides an electronic device, including:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the object association method of any embodiment of the present application.

In a fourth aspect, the embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the program, when executed by a processor, implements the target association method described in any embodiment of the present application.

The embodiment of the application provides a target association method, a target association device, electronic equipment and a storage medium, wherein at least one object is respectively identified through a first sensor and a second sensor, the at least one object identified by the first sensor is used as a reference target, and the position information of each reference target in the at least one reference target is obtained; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target; setting a corresponding association threshold for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target; setting a corresponding association threshold and position information of each target in at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated; and associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table. According to the method and the device, the accuracy of target association can be improved, and the real-time requirement can be met by reducing the calculated amount during target association.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

fig. 1A is a first flowchart of a target association method according to an embodiment of the present disclosure;

fig. 1B is a schematic diagram of target association of a target association method provided in an embodiment of the present application;

fig. 2 is a second flowchart of a target association method according to an embodiment of the present application;

fig. 3 is a third flow chart of a target association method according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a target association apparatus according to an embodiment of the present application;

fig. 5 is a block diagram of an electronic device for implementing the target association method of the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Example one

Fig. 1A is a first flowchart of a target association method according to an embodiment of the present disclosure; fig. 1B is a schematic view of target association of a target association method provided in an embodiment of the present application. The embodiment can be applied to the situation that whether the targets identified by different sensors are the same target or not is judged, and if the targets are the same target, the targets are associated. The object associating method provided in this embodiment may be executed by an object associating apparatus provided in this embodiment, where the apparatus may be implemented by software and/or hardware and integrated in an electronic device executing the method.

Referring to fig. 1A, the method of the present embodiment includes, but is not limited to, the following steps:

s110, respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; and taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target.

The first sensor and the second sensor are any sensors with identification functions, optionally, the first sensor is a vision sensor, and the second sensor is a millimeter wave radar sensor. The target may be any object in the identification area, such as a walking person, a moving vehicle, a stationary roadside marker, or the like. The coordinate system of the position information may be a world coordinate system (such as latitude and longitude information), or may be a two-dimensional coordinate system or a three-dimensional coordinate system with the respective sensors as coordinate centers.

In the embodiment of the application, at least one object in the same identification area is identified by the first sensor and the second sensor respectively, and the identified data is uploaded to the processor. The processor processes the data and associates the object identified by the first sensor with the object identified by the second sensor if the object identified by the first sensor and the object identified by the second sensor belong to the same object. The processor may be a processor configured for a vehicle that integrates the first sensor and the second sensor, or may be a processor in a cloud server.

Because different sensors have different recognition capabilities for objects, one reference sensor (such as the first sensor) is selected from the first sensor and the second sensor according to human experience, namely, the object recognized by the reference sensor is taken as a standard and is associated with the object recognized by the other sensor. Specifically, a first sensor is used as a reference sensor, at least one object identified by the first sensor is used as a reference target, and at least one object identified by a second sensor is used as a target. Associating the at least one reference target with the at least one target by respectively obtaining the position information of each of the at least one reference target and the position information of each of the at least one target.

Optionally, if the position information of the at least one reference target and the position information of the at least one target are not in the same coordinate system, the position information of the at least one reference target and the position information of the at least one target are respectively converted into position information in the same coordinate system.

In the embodiment of the present application, when the coordinate system of the position information is a two-dimensional coordinate system or a three-dimensional coordinate system with respective sensors as coordinate centers, the obtained position information of the reference target and the position information of the target are not in the same coordinate system due to different installation positions and installation angles of different sensors, and then the position information of the reference target and the position information of the target need to be respectively converted into position information in the same coordinate system, such as a world coordinate system, a vehicle coordinate system, and the like. This has the advantage that the position information of the different sensors can be synchronized spatially.

In this embodiment, since the frequency of uploading data to the processor by different sensors is not used, the processor needs to compensate the position information of the first sensor and the second sensor by a pre-trained model according to the time stamp of receiving data of each sensor, so that the position information of different sensors is synchronized in time.

Specifically, according to the recognition time of the at least one reference target and the recognition time of the at least one target, the position information of each reference target in the at least one reference target and the position information of each target in the at least one target in the same coordinate system are respectively corrected through a pre-trained model, and the corrected position information of the at least one reference target and the corrected position information of the at least one target are obtained.

S120, setting corresponding association thresholds for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target.

Wherein the associated threshold comprises a threshold shape and a threshold size. The shape of the threshold is not specifically limited, and a circular threshold, an elliptical threshold and a rectangular threshold can be selected, and a variable rectangular threshold, a threshold size, a length value and a width value are preferred. The included position information includes at least a lateral distance and a longitudinal distance.

Optionally, the length value and the width value of the variable rectangular threshold corresponding to each of the at least one reference target are determined according to the longitudinal distance of each of the at least one reference target.

In the embodiment of the application, as the longitudinal distance of the reference target identified by the first sensor increases, the length value and the width value of the variable rectangular threshold of the reference target also increase. According to the method, the target can be prevented from being lost by adopting the variable rectangular threshold according to the actual distribution characteristics of the sensors, and the accuracy of target association in the following steps can be improved. The calculation formula (1) of the length value and the width value of the variable rectangular threshold is shown as follows:

wherein, L is the length of the variable rectangular threshold; w is the width of the variable rectangular threshold; a is₁、b₁、a₂And b₂Is a coefficient set empirically; i is a reference target; x is the longitudinal distance of the reference target.

As shown in fig. 1B, the vertical axis represents the longitudinal distance in the vehicle coordinate system, the horizontal axis represents the lateral distance in the vehicle coordinate system, the circle represents the reference target (e.g., V1, V2, …, V6) identified by the first sensor, the pentagram represents the target (e.g., R1, R2, …, R8) identified by the second sensor, and the box represents the variable rectangular gate of the reference target, and it can be seen that the gate size of the variable rectangular gate of the reference target increases as the longitudinal distance of the reference target increases.

S130, setting corresponding association thresholds and position information of each target in at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated.

In this embodiment of the present application, after the above steps are performed, after the corresponding association threshold is set for each reference target in the at least one reference target, based on the corresponding association threshold set for each reference target and the position information of each target in the at least one target, the targets that are not within any association threshold are removed from the at least one target, and the remaining targets in the at least one target are taken as the targets to be associated, that is, the targets that fall within any association threshold may be the targets to be associated of a certain reference target.

As shown in fig. 1B, R1, R7, and R8 in the target (i.e., the five-pointed star in the figure) may be clutter; or R1, R7, and R8 are indeed true targets, but since the first sensor does not recognize R1, R7, and R8, it is not necessary to associate R1, R7, and R8 with the reference target when the targets are associated. Therefore, R1, R7 and R8 are eliminated before the target association of the at least one reference target with the at least one target. After the processing of this step, the targets to be associated are R2, R3, …, R6 with reference to the targets V1, V2, …, V6.

In the embodiment of the application, each reference target in the at least one reference target is not directly associated with each target in the at least one target, but the at least one target is filtered by adopting a variable rectangular threshold to obtain the target to be associated. The advantage of this arrangement is that not only can clutter interference of the second sensor be eliminated, but also the difficulty of target association (the dimension of the association matrix in the following steps) can be greatly reduced, and the amount of calculation is reduced because the amount of calculation increases exponentially due to the dimension of the association matrix.

S140, associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table.

The association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

In the embodiment of the application, through the above steps, the objects which are not within any association threshold are removed from the at least one object, and the remaining objects in the at least one object are used as the objects to be associated. Then, based on a maximum weight matching algorithm, solving an incidence matrix of at least one reference target and at least one target to be associated, and associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table.

According to the technical scheme provided by the embodiment, at least one object is respectively identified through a first sensor and a second sensor, the at least one object identified by the first sensor is used as a reference target, and the position information of each reference target in the at least one reference target is obtained; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target; setting a corresponding association threshold for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target; setting a corresponding association threshold and position information of each target in at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated; and associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table. The method and the device for detecting the target loss can solve the problem that the target is lost in the prior art by adopting the variable rectangular threshold. According to the method and the device, before the target association is carried out on at least one reference target and at least one target, the variable rectangular threshold is adopted to filter the at least one target, so that clutter interference of the second sensor can be eliminated, and the difficulty of the target association can be greatly reduced. By executing the technical scheme of the application, the accuracy of target association can be improved, and the real-time requirement can be met by reducing the calculated amount during target association.

Example two

Fig. 2 is a second flowchart of the target association method according to the embodiment of the present application. The embodiment of the application is optimized on the basis of the embodiment, and specifically optimized as follows: a detailed explanation of the screening process of the associated pair is added.

Referring to fig. 2, the method of the present embodiment includes, but is not limited to, the following steps:

s210, respectively identifying at least one object through a first sensor and a second sensor, taking the at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; and taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target.

S220, setting corresponding association thresholds for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target.

S230, setting corresponding association thresholds and position information of each target in at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated.

S240, associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table.

S250, acquiring the speed information of each reference target in the at least one reference target and the speed information of each target to be associated in the at least one target to be associated.

In the embodiment of the application, when the first sensor and the second sensor respectively identify the at least one reference target and the at least one target, not only the position information of each reference target and the position information of each target are acquired, but also the speed information of each reference target and the speed information of each target need to be acquired.

Optionally, information of each reference target and each type of target may also be obtained, for example: people, vehicles, roadside markers, and the like.

S260, calculating the speed difference between the reference target and the target to be correlated in each correlation pair in the correlation relation table according to the speed information of each reference target in the at least one reference target and the speed information of each target to be correlated in the at least one target to be correlated.

In the embodiment of the present application, an association table is obtained through step S240. The association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target. The speed difference between the reference target and the target to be correlated in each correlation pair needs to be calculated.

And S270, if the speed difference is not within the preset threshold range, removing the association pair from the association relation table.

In the embodiment of the application, if the speed difference between the reference target and the target to be associated in a certain association pair is not within the preset threshold range, which indicates that the reference target and the target to be associated may not be an object, the association pair is removed from the association relation table; if the speed difference is within the preset threshold range, the reference target and the target to be correlated are indicated to be used as one object, and the correlation pair is reserved.

The reference target and the target to be correlated in the correlation pair are filtered by adopting a speed difference threshold, namely the reference target and the target to be correlated in the correlation pair are not only closest to each other in space, but also the speed difference meets a preset threshold range, so that the moving vehicle target is prevented from being correlated with the static radar fence target. For example, as shown in fig. 1B, if V1 is associated with R3 and V2 has no associated target, the association pair of V1 and R3 can be eliminated by determining that the speed difference between V1 and R3 does not satisfy the preset threshold range.

According to the technical scheme provided by the embodiment, at least one object identified by a first sensor is used as a reference target, and the position information and the speed information of each reference target in the at least one reference target are acquired; taking at least one object identified by the second sensor as a target, and acquiring position information and speed information of each target in the at least one target; setting a corresponding association threshold for each reference target in at least one reference target; removing targets which are not in any association threshold from at least one target, and taking each residual target in the at least one target as a target to be associated; associating each reference target in the at least one reference target with each target to be associated in the at least one target to be associated to obtain an association relation table; calculating the speed difference between the reference target and the target to be associated in each association pair in the association relation table according to the speed information of each reference target in the at least one reference target and the speed information of each target to be associated in the at least one target to be associated; and if the speed difference is not within the preset threshold range, removing the association pair from the association relation table. By setting the speed difference threshold, the correlation pairs which do not conform to the speed difference are eliminated from the correlation table, and the accuracy of target correlation between the reference target and the target to be correlated can be ensured.

EXAMPLE III

Fig. 3 is a third flow chart of the target association method according to the embodiment of the present application. The embodiment of the application is optimized on the basis of the embodiment, and specifically optimized as follows: a detailed explanation of the process of performing object association between the reference object and the object to be associated is added.

Referring to fig. 3, the method of the present embodiment includes, but is not limited to, the following steps:

s310, identifying at least one object through a first sensor and a second sensor respectively, taking the at least one object identified by the first sensor as a reference target, and acquiring position information of each reference target in the at least one reference target; and taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target.

S320, setting corresponding association thresholds for each reference target in the at least one reference target according to the position information of each reference target in the at least one reference target.

S330, setting corresponding association thresholds and position information of each target in at least one target based on each reference target, removing targets which are not in any association threshold from the at least one target, and taking each residual target in the at least one target as a target to be associated.

S340, establishing a correlation matrix between the at least one reference target and the at least one target to be correlated according to the position information of each reference target in the at least one reference target and the position information of each target to be correlated in the at least one target to be correlated.

In the embodiment of the application, a correlation matrix T between at least one reference target and at least one target to be correlated is calculated, and each element T in the matrix T_i,jIs shown in the formula (2):

wherein i represents a reference target; j represents the object to be associated; t is_i,jRepresenting the similarity between the ith reference target and the jth target to be associated (mayTo express similarity by mahalanobis distance), when the matrix T is_i,jThe smaller the value of (a) is, the more likely it is that the reference target and the target to be associated are the same object; v. of_iAn observation vector representing a reference target identified by the first sensor; r is_jAn observation vector representing the target to be associated identified by the second sensor; p_vi、P_rjRepresenting the covariance matrices of the reference target and the target to be correlated, respectively.

Optionally, after the association matrix between the at least one reference object and the at least one object to be associated is established, if the number of the at least one reference object is inconsistent with the number of the at least one object to be associated, the association matrix is filled, so that the association matrix is a square matrix.

In the embodiment of the application, since the maximum weight matching algorithm requires that the association matrix T is a square matrix, when the number of at least one reference target is inconsistent with the number of at least one target to be associated (e.g., i ≠ j), the dimension n of the T matrix is set to max (i, j), the association matrix is filled by adding a virtual target, and the virtual target is assigned with infinity, so that the association matrix T is a square matrix.

And S350, transforming the incidence matrix according to a preset transformation rule to obtain a transformed incidence matrix, and calculating the transformed incidence matrix to obtain an incidence relation table.

In the embodiment of the present application, the modeling problem in step S340 is a minimum value problem, and since the maximum weight matching algorithm is a maximum value problem, the correlation matrix is transformed, and the minimum value problem is converted into a maximum value problem. Optionally, taking reciprocal normalization on elements of the incidence matrix; it is preferable to take the inverse number of the elements of the correlation matrix, because taking the inverse number makes the weight between the targets too dense, which easily causes a problem of erroneous correlation between the reference target and the target to be correlated. Exemplarily, as in fig. 1B, associating V1 with R3 (V1 should be associated with R2), an association error occurs.

In the embodiment of the present application, the correlation matrix after transformation is calculated to obtain an optimization function as shown in formula (3):

wherein i represents a reference target; j represents the object to be associated; n represents the number of reference targets or targets to be associated; t is_i,jRepresenting the similarity between the ith reference target and the jth target to be associated; h (i, j) represents an association relation table, the value of H (i, j) is 0 or 1, when the value is 0, the ith reference target and the jth target to be associated are not the same object, and when the value is 1, the ith reference target and the jth target to be associated are the same object.

According to the technical scheme provided by the embodiment, at least one object identified by a first sensor is used as a reference target, and the position information of each reference target in the at least one reference target is obtained; taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target; setting a corresponding association threshold for each reference target in at least one reference target; removing targets which are not in any association threshold from at least one target, and taking each residual target in the at least one target as a target to be associated; establishing a correlation matrix between at least one reference target and at least one target to be correlated according to the position information of each reference target in at least one reference target and the position information of each target to be correlated in at least one target to be correlated; and transforming the incidence matrix according to a preset transformation rule to obtain a transformed incidence matrix, and calculating the transformed incidence matrix to obtain an incidence relation table. By executing the technical scheme of the application, the accuracy of target association can be improved, and the real-time requirement can be met by reducing the calculated amount during target association.

Example four

Fig. 4 is a schematic structural diagram of a target association apparatus according to an embodiment of the present application, and as shown in fig. 4, the apparatus 400 may include:

an information obtaining module 410, configured to respectively identify at least one object through a first sensor and a second sensor, use the at least one object identified by the first sensor as a reference target, and obtain position information of each reference target in the at least one reference target; and taking at least one object identified by the second sensor as a target, and acquiring position information of each target in the at least one target.

A threshold setting module 420, configured to set a corresponding association threshold for each reference target of the at least one reference target according to the position information of each reference target of the at least one reference target.

The target determining module 430 is configured to set a corresponding association threshold and position information of each target of the at least one target based on each reference target, remove targets that are not within any association threshold from the at least one target, and use each remaining target of the at least one target as a target to be associated.

The target association module 440 is configured to associate each reference target of the at least one reference target with each target to be associated of the at least one target to be associated, so as to obtain an association relationship table; the association relation table comprises at least one association pair, and each association pair comprises a reference target and a target to be associated having an association relation with the reference target.

Optionally, the association threshold is a variable rectangular threshold; the position information includes at least a lateral distance and a longitudinal distance.

Further, the threshold setting module 420 is specifically configured to determine, according to the longitudinal distance of each reference target of the at least one reference target, a length value and a width value of the variable rectangular threshold corresponding to each target of the at least one reference target.

Further, the target associating apparatus may further include: the associative pair determination module 450 (not shown in the figure);

the association pair determining module 450 is configured to obtain speed information of each reference target of the at least one reference target and speed information of each target to be associated of the at least one target to be associated; calculating the speed difference between the reference target and the target to be associated in each association pair in the association relation table according to the speed information of each reference target in the at least one reference target and the speed information of each target to be associated in the at least one target to be associated; and if the speed difference is not within the range of the preset threshold value, removing the association pair from the association relation table.

Further, the object associating module 440 includes: an incidence matrix establishing unit and an incidence relation table determining unit;

the association matrix establishing unit is configured to establish an association matrix between the at least one reference target and the at least one object to be associated according to the position information of each reference target in the at least one reference target and the position information of each object to be associated in the at least one object to be associated.

And the incidence relation table determining unit is used for transforming the incidence matrix according to a preset transformation rule to obtain a transformed incidence matrix and calculating the transformed incidence matrix to obtain an incidence relation table.

Further, the target correlation module 440 further includes a correlation matrix filling unit;

the incidence matrix filling unit is configured to, after the incidence matrix between the at least one reference target and the at least one object to be correlated is established, fill the incidence matrix if the number of the at least one reference target is inconsistent with the number of the at least one object to be correlated, so that the incidence matrix is a square matrix.

Further, the target associating apparatus may further include: a coordinate system conversion module 460 (not shown);

the coordinate system converting module 460 is configured to, after obtaining the position information of each of the at least one reference target and obtaining the position information of each of the at least one target, if the position information of the at least one reference target and the position information of the at least one target are not in the same coordinate system, respectively convert the position information of the at least one reference target and the position information of the at least one target into position information in the same coordinate system.

Further, the target associating apparatus may further include: an information correction module 470 (not shown in the figure);

the coordinate system conversion module 470 is configured to, after acquiring the position information of each of the at least one reference target and the position information of each of the at least one target, correct, according to the recognition time of the at least one reference target and the recognition time of the at least one target, the position information of each of the at least one reference target and the position information of each of the at least one target in the same coordinate system through a pre-trained model, and obtain the corrected position information of the at least one reference target and the corrected position information of the at least one target.

The object association apparatus provided by this embodiment is applicable to the object association method provided by any of the above embodiments, and has corresponding functions and advantageous effects.

EXAMPLE five

Fig. 5 is a block diagram of an electronic device adapted to implement the target association method of an embodiment of the present application, and fig. 5 shows a block diagram of an exemplary electronic device adapted to implement embodiments of the present application. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application. The electronic device can be a smart phone, a tablet computer, a notebook computer, a vehicle-mounted terminal, a wearable device and the like.

As shown in fig. 5, the electronic device 500 is embodied in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to: one or more processors or processing units 516, a memory 528, and a bus 518 that couples the various system components including the memory 528 and the processing unit 516.

Bus 518 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Electronic device 500 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by electronic device 500 and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 528 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)530 and/or cache memory 532. The electronic device 500 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 534 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 518 through one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

A program/utility 540 having a set (at least one) of program modules 542, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in, for example, the memory 528, each of which examples or some combination may include an implementation of a network environment. Program modules 542 generally perform the functions and/or methods of the embodiments described herein.

The electronic device 500 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), with one or more devices that enable a user to interact with the electronic device 500, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 500 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 522. Also, the electronic device 500 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 520. As shown in FIG. 5, the network adapter 520 communicates with the other modules of the electronic device 500 via the bus 518. It should be appreciated that although not shown in FIG. 5, other hardware and/or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 516 executes various functional applications and data processing by executing programs stored in the memory 528, for example, to realize the target association method provided by any embodiment of the present application.

EXAMPLE six

A sixth embodiment of the present application further provides a computer-readable storage medium, on which a computer program (or referred to as computer-executable instructions) is stored, where the computer program, when executed by a processor, can be used to execute the object association method provided in any of the above embodiments of the present application.

The computer storage media of the embodiments of the present application may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for embodiments of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the embodiments of the present application have been described in more detail through the above embodiments, the embodiments of the present application are not limited to the above embodiments, and many other equivalent embodiments may be included without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.