阅读说明：本技术 位姿评价方法、位姿确定方法、对应装置和电子设备 (Pose evaluation method, pose determination method, corresponding device and electronic equipment ) 是由 冯洋 于 2021-08-13 设计创作，主要内容包括：本申请提供一种位姿评价方法、位姿确定方法、对应装置和电子设备,该方法的一具体实施方式包括：获取待处理图像；所述待处理图像包括目标对象的图像；根据所述待处理图像,获取所述目标对象所对应的目标模型、以及所述目标对象在所述待处理图像的拍摄时刻所对应的估计位姿；确定所述目标模型在所述估计位姿下投影在所述待处理图像上的投影点的凸包；基于所述投影点的凸包以及所述目标对象的图像,计算所述估计位姿的置信度,以根据所述置信度评价所述估计位姿。该方法可以计算估计位姿的置信度,以能够根据得到的置信度对估计位姿进行评价,确定出较为准确的目标位姿。(The application provides a pose evaluation method, a pose determination method, a corresponding device and electronic equipment, wherein a specific implementation mode of the method comprises the following steps: acquiring an image to be processed; the image to be processed comprises an image of a target object; according to the image to be processed, acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting time of the image to be processed; determining a convex hull of a projection point of the target model projected on the image to be processed under the estimation pose; calculating a confidence of the estimated pose based on the convex hull of the projection points and the image of the target object to evaluate the estimated pose according to the confidence. The method can calculate the confidence coefficient of the estimated pose so as to evaluate the estimated pose according to the obtained confidence coefficient and determine the more accurate target pose.)

1. A pose evaluation method is characterized by comprising:

acquiring an image to be processed; the image to be processed comprises an image of a target object;

according to the image to be processed, acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting time of the image to be processed;

determining a convex hull of a projection point of the target model projected on the image to be processed under the estimation pose;

calculating a confidence of the estimated pose based on the convex hull of the projection points and the image of the target object to evaluate the estimated pose according to the confidence.

2. The method of claim 1, wherein the calculating a confidence level for the estimated pose based on the convex hull of the proxels and the image of the target object comprises:

determining a contour polygon of the target object in the image to be processed; the outline polygon represents a polygon formed by the outline lines of the target object;

and calculating the fit degree between the outline polygon and the convex hull, and determining the fit degree as the confidence coefficient.

3. The method of claim 1, wherein the calculating a confidence level for the estimated pose based on the convex hull of the proxels and the image of the target object comprises:

determining a first fit degree between a convex hull of the projection point and a minimum bounding rectangle of the object corresponding to the target object; and

determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object;

calculating the confidence based on the first fit degree and the second fit degree.

4. The method of claim 3, wherein determining the first degree of fit between the convex hull of the proxel and the object minimum bounding rectangle to which the target object corresponds comprises:

determining a first intersection portion between the convex hull of the proxel and the object minimum bounding rectangle;

characterizing the first fit degree by a first area ratio between the first intersection portion and a convex hull of the projection point; and

the determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object includes:

determining a second intersection portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle;

determining a union part between the convex hull minimum bounding rectangle and the object minimum bounding rectangle;

characterizing the second fit by a second area ratio between the second intersection portion and the union portion; and

determining the confidence level based on the first fit level and the second fit level includes:

calculating the confidence based on the first area ratio and the second area ratio.

5. The method of claim 4, wherein calculating the confidence based on the first area ratio value and the second area ratio value comprises:

and multiplying the first area ratio and the second area ratio to calculate the confidence.

6. The method according to any one of claims 3-5, wherein the object minimum bounding rectangle is obtained online by detecting the image to be processed by a target detection algorithm; or the target object is obtained offline through labeling.

7. A pose determination method, comprising:

obtaining confidence of the estimated pose; the confidence is calculated by the pose evaluation method as set forth in any one of claims 1 to 6;

and when the confidence coefficient is detected to meet the evaluation condition, determining the estimated pose as the target pose of the target object.

8. The method of claim 7, wherein determining the estimated pose as the target pose of the target object upon detecting that the confidence level satisfies an evaluation condition comprises:

and when detecting that the numerical value corresponding to the confidence coefficient is larger than a confidence coefficient threshold value, determining the estimated pose as the target pose of the target object.

9. A pose evaluation apparatus characterized by comprising:

the first acquisition module is used for acquiring an image to be processed; the image to be processed comprises an image of a target object;

the second acquisition module is used for acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting moment of the image to be processed according to the image to be processed;

a first determination module, configured to determine a convex hull of a projection point of the target model projected on the image to be processed in the estimated pose;

a calculation module, configured to calculate a confidence of the estimated pose based on the convex hull of the projection point and the image of the target object, so as to evaluate the estimated pose according to the confidence.

10. A pose determination apparatus, characterized by comprising:

the third acquisition module is used for acquiring the confidence coefficient of the estimated pose; the confidence is obtained by the pose evaluation apparatus according to claim 9;

and the second determining module is used for determining the estimated pose as the target pose of the target object when the confidence coefficient is detected to meet the evaluation condition.

11. An electronic device comprising a processor and a memory, the memory storing computer readable instructions that, when executed by the processor, perform the method of any of claims 1-6 or 7-8.

12. A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6 or 7-8.

Technical Field

The present application relates to the field of image processing, and in particular, to a pose evaluation method, a pose determination method, a corresponding apparatus, and an electronic device.

Background

In the field of image processing, it is necessary to accurately obtain pose information of an object, and the determined pose of the object can be used in the fields of augmented reality, unmanned driving and the like.

The following methods exist in the related art to obtain the pose of an object: (1) firstly, detecting two-dimensional key points of a target object in an image to be processed, then determining projection points of a plurality of three-dimensional model points corresponding to a three-dimensional model of the target object in the image to be processed, and finally obtaining pose parameters of the target object based on the minimum re-projection errors between the plurality of projection points and the corresponding two-dimensional key points; (2) and (5) regressing the pose parameters and the confidence coefficient of the object by utilizing the neural network model. However, when the pose parameter is obtained by the method (1), the problem that the re-projection error is small but the pose parameter is not matched with the actual pose exists; the confidence obtained by the method (2) has an error. Therefore, neither of the two methods can accurately evaluate the pose parameters.

Disclosure of Invention

An object of the embodiments of the present application is to provide a pose evaluation method, a pose determination method, a corresponding apparatus, and an electronic device, which are used to calculate a confidence of an estimated pose, so that the estimated pose can be evaluated according to the obtained confidence, and a more accurate target pose can be determined.

In a first aspect, an embodiment of the present application provides a pose estimation method, where the method includes: acquiring an image to be processed; the image to be processed comprises an image of a target object; according to the image to be processed, acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting time of the image to be processed; determining a convex hull of a projection point of the target model projected on the image to be processed under the estimation pose; calculating a confidence of the estimated pose based on the convex hull of the projection points and the image of the target object to evaluate the estimated pose according to the confidence. Therefore, the confidence coefficient of the estimated pose can be calculated, and the estimated pose can be evaluated more accurately.

Optionally, the calculating a confidence of the estimated pose based on the convex hull of the proxels and the image of the target object comprises: determining a contour polygon of the target object in the image to be processed; the outline polygon represents a polygon formed by the outline lines of the target object; and calculating the fit degree between the outline polygon and the convex hull, and determining the fit degree as the confidence coefficient. Here, an embodiment of calculating the confidence level is provided, and the calculated confidence level can be used for evaluating the estimated pose more accurately.

Optionally, the calculating a confidence of the estimated pose based on the convex hull of the proxels and the image of the target object comprises: determining a first fit degree between a convex hull of the projection point and a minimum bounding rectangle of the object corresponding to the target object; determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object; calculating the confidence based on the first fit degree and the second fit degree. Therefore, the confidence coefficient of the estimated pose can be calculated based on the determined first fitting degree and the second fitting degree, and on the basis of effectively saving cost investment, the accurate confidence coefficient can be obtained.

Optionally, the determining a first degree of fitting between the convex hull of the projection point and the minimum bounding rectangle of the object corresponding to the target object includes: determining a first intersection portion between the convex hull of the proxel and the object minimum bounding rectangle; characterizing the first fit degree by a first area ratio between the first intersection portion and a convex hull of the projection point; and the determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object comprises: determining a second intersection portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle; determining a union part between the convex hull minimum bounding rectangle and the object minimum bounding rectangle; characterizing the second fit by a second area ratio between the second intersection portion and the union portion; and determining the confidence level based on the first fit level and the second fit level comprises: calculating the confidence based on the first area ratio and the second area ratio. Here, an embodiment is provided for calculating a confidence level based on a first fit level and a second fit level.

Optionally, calculating the confidence based on the first area ratio and the second area ratio comprises: and multiplying the first area ratio and the second area ratio to calculate the confidence coefficient so as to obtain a specific numerical value of the confidence coefficient.

Optionally, the minimum bounding rectangle of the object is obtained on line by detecting the image to be processed through a target detection algorithm; or the target object is obtained offline through labeling. Therefore, the minimum circumscribed rectangle of the object can be obtained according to actual conditions in specific application.

In a second aspect, an embodiment of the present application provides a pose determination method, where the method includes: obtaining confidence of the estimated pose; the confidence coefficient is calculated by the pose evaluation method in the first aspect; and when the confidence coefficient is detected to meet the evaluation condition, determining the estimated pose as the target pose of the target object.

Optionally, the determining the estimated pose as the target pose of the target object when detecting that the confidence coefficient satisfies the evaluation condition includes: and when detecting that the numerical value corresponding to the confidence coefficient is larger than a confidence coefficient threshold value, determining the estimated pose as the target pose of the target object. Here, an embodiment is provided in which the estimated pose can be determined as the target pose.

In a third aspect, an embodiment of the present application provides a pose evaluation apparatus, including: the first acquisition module is used for acquiring an image to be processed; the image to be processed comprises an image of a target object; the second acquisition module is used for acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting moment of the image to be processed according to the image to be processed; a first determination module, configured to determine a convex hull of a projection point of the target model projected on the image to be processed in the estimated pose; a calculation module, configured to calculate a confidence of the estimated pose based on the convex hull of the projection point and the image of the target object, so as to evaluate the estimated pose according to the confidence.

In a fourth aspect, an embodiment of the present application provides a pose determination apparatus, including: the third acquisition module is used for acquiring the confidence coefficient of the estimated pose; the confidence is obtained by the pose evaluation device according to the third aspect; and the second determining module is used for determining the estimated pose as the target pose of the target object when the confidence coefficient is detected to meet the evaluation condition.

In a fifth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method as provided in the first aspect are executed.

In a sixth aspect, embodiments of the present application provide a readable storage medium, on which a computer program is stored, where the computer program runs the steps in the method provided in the first aspect when executed by a processor.

Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a flowchart of a pose evaluation method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application;

fig. 3 is a flowchart of another pose evaluation method provided in the embodiment of the present application;

fig. 4 is a flowchart of a pose determination method provided in an embodiment of the present application;

fig. 5 is a structural block diagram of a pose evaluation apparatus provided in an embodiment of the present application;

fig. 6 is a block diagram of a pose determination apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device for executing a pose evaluation method according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

In recent years, technical research based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition, has been actively developed. Artificial Intelligence (AI) is an emerging scientific technology for studying and developing theories, methods, techniques and application systems for simulating and extending human Intelligence. The artificial intelligence subject is a comprehensive subject and relates to various technical categories such as chips, big data, cloud computing, internet of things, distributed storage, deep learning, machine learning and neural networks. Computer vision is used as an important branch of artificial intelligence, particularly a machine is used for identifying the world, and the computer vision technology generally comprises the technologies of face identification, living body detection, fingerprint identification and anti-counterfeiting verification, biological feature identification, face detection, pedestrian detection, target detection, pedestrian identification, image processing, image identification, image semantic understanding, image retrieval, character identification, video processing, video content identification, behavior identification, three-dimensional reconstruction, virtual reality, augmented reality, synchronous positioning and map construction (SLAM), computational photography, robot navigation and positioning and the like. With the research and progress of artificial intelligence technology, the technology is applied to various fields, such as security, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone images, cloud services, smart homes, wearable equipment, unmanned driving, automatic driving, smart medical treatment, face payment, face unlocking, fingerprint unlocking, testimony verification, smart screens, smart televisions, cameras, mobile internet, live webcasts, beauty treatment, medical beauty treatment, intelligent temperature measurement and the like.

In the related technology, the problem that whether the estimated pose parameters are credible cannot be accurately evaluated exists; in order to solve the problem, the application provides a pose evaluation method; further, the pose evaluation method determines a convex hull of a projection point of a target model of the target object on the image to be processed, and calculates the confidence coefficient of the estimated pose by using the convex hull and the image of the target object, so that whether the estimated pose is credible can be evaluated by using the confidence coefficient. In this way, the convex hull of the projection point is obtained by projecting the target model on the image to be processed under the estimation pose, and if the estimation pose is the same as the actual pose of the target object at the moment when the image to be processed is shot, the convex hull of the projection point should coincide with the image edge of the target object. Therefore, the confidence coefficient obtained by the method can be used for evaluating the estimated pose more accurately, and the problems are solved. In practice, the pose evaluation method can be applied to corresponding pose estimation processes such as vehicles, unmanned planes and the like to evaluate the estimated pose. The pose evaluation method is exemplarily explained in the embodiment of the present application by taking a pose estimation process applied to a vehicle as an example. That is, the target object may include a target vehicle.

The above solutions in the related art are all the results of practical and careful study of the inventor, and therefore, the discovery process of the above problems and the solutions proposed by the following embodiments of the present invention to the above problems should be the contribution of the inventor to the present invention in the course of the present invention.

Please refer to fig. 1, which illustrates a flowchart of a pose evaluation method according to an embodiment of the present application. As shown in fig. 1, the pose evaluation method may include the following steps 101 to 104.

Step 101, acquiring an image to be processed; the image to be processed comprises an image of a target object;

the target object may include, for example, a target truck, a target van, a target car, and the like.

In some application scenarios, the image to be processed may be acquired by a method such as shooting or intercepting a target image in a target video by a camera. The image to be processed may include an image of the target object, and may also include images of other objects.

102, acquiring a target model corresponding to the target object and an estimated pose of the target object at the shooting time of the image to be processed according to the image to be processed;

after the image to be processed is acquired, a target model corresponding to the target object may be further acquired. In some application scenarios, the target model may be obtained by matching in a preset model library after identifying a target vehicle of the image to be processed. The preset model library may include, for example, vehicle models respectively corresponding to a minibus, a car, a truck, and the like, and the vehicle parameters of the vehicle models may include, for example, vehicle parameters obtained from a manufacturer or market experience. For example, the vehicle parameters of the truck may include a height of 2.5 meters, a length of 9 meters, a width of 2.3 meters, and a relative positional relationship between key points on the truck, such as a logo, a license plate, a rearview mirror, etc. In the application scenes, a target model corresponding to a target vehicle can be obtained by identifying the target vehicle in the image to be processed, and then the length, the width, the height, the relative position relationship among model points and other information of the target vehicle can be determined through the target model.

In some application scenarios, the estimated pose of the target object corresponding to the shooting time of the image to be processed can be further obtained. The estimated pose can be characterized by using pose parameters of the target object in an actual application scene. The pose parameters can be characterized by coordinate information (x, y, z) of the target object in the world coordinate system, for example. Further, the coordinate parameters "x", "y", and "z" may be any corresponding values in the world coordinate system, where "x" may represent horizontal axis coordinate information, "y" may represent vertical axis coordinate information, and "z" may represent vertical axis coordinate information. In these application scenarios, the estimated pose may be obtained, for example, through a Pixel-wise Voting Network (PVNet).

103, determining a convex hull target model of a projection point of the target model projected on the image to be processed under the estimation pose;

after the target model corresponding to the target object is determined, a plurality of projection points of a plurality of model points on the target model projected on the image to be processed under the estimated pose can be determined. In some application scenarios, for example, a plurality of model points that constitute the contour of the target model may be selected, and the selected plurality of model points may be projected onto the image to be processed to obtain a corresponding plurality of projection points. In other application scenarios, as many model points as possible may be selected to facilitate determination of a more accurate convex hull of the projected points. In other application scenarios, model points corresponding to a plurality of two-dimensional key points detected in the image to be processed may be selected for projection. The two-dimensional key points can be used for representing the position of the target object in the actual application scene.

In some application scenarios, after determining a plurality of model points, projection points of the model points in the image to be processed may be determined based on a projection formula. The projection formula herein may include, for example:wherein, the lambda is a scale factor,is an internal parameter matrix of the camera and is,x, Y, Z is the coordinate parameter of model point, u and v are the pixel coordinate parameter of projection point, u₀、v₀Is the center of the image plane. In these application scenarios, the scale factor, the internal parameters of the camera, and the external parameters can be regarded as known quantities, which can be obtained by using the prior art, and are not described herein.

After obtaining a plurality of proxels, the convex hull of the proxels may be determined. That is, a convex polygon formed by connecting outermost projection points is determined based on a plurality of projection points, and the determined convex polygon can be regarded as a convex hull of the projection points. For example, in the vehicle application scene diagram shown in fig. 2, if a plurality of projection points 1 are obtained, a convex hull 2 of the corresponding projection points can be determined.

And 104, calculating the confidence coefficient of the estimated pose based on the convex hull of the projection point and the image of the target object, and evaluating the estimated pose according to the confidence coefficient.

In some application scenarios, after determining the convex hull of the proxel, a confidence level of the estimated pose may be calculated based on the convex hull of the proxel and the image of the target object.

The above-described confidence may be regarded as a confidence that the estimated pose can be regarded as the actual pose of the target object at the time when the image to be processed is captured.

The image of the target object may be regarded as an image represented by an image area occupied by the target object in the image to be processed. For example, the road surface and the target vehicle a are included in the image to be processed, and at this time, the image area occupied by the target vehicle a may be regarded as the image of the target object.

Through the steps 101 to 104, the confidence of the estimated pose can be calculated, so that the estimated pose can be evaluated more accurately.

In some optional implementations, the step 104 may include: determining a contour polygon of the target object in the image to be processed; the outline polygon represents a polygon formed by the outline lines of the target object; and calculating the fit degree between the outline polygon and the convex hull, and determining the fit degree as the confidence coefficient.

In the related art, if the estimated pose is the same as the actual pose, a polygon formed by the convex hull of the projection point and the contour line of the target object should coincide. Therefore, the estimated pose can be evaluated by the degree of fit between the convex hull of the projection point and the outline polygon of the target object.

In some application scenarios, the degree of fit between the outline polygon and the convex hull may be determined, for example, by determining a straight-line distance between a projection point on the convex hull of the projection points and a corresponding outline point on the outline line.

In these application scenarios, the contour lines may be obtained based on manual labeling, and in the case where the convex hull of the proxels is determined, the confidence may be calculated in an offline state. In other application scenarios, the contour lines may also be detected online, for example, by an edge detection algorithm. Therefore, the confidence may also be calculated in an online state.

In some application scenarios, since the projection points on the convex hull are already determined, the confidence of the estimated pose can also be determined by comparing the projection points with the corresponding two-dimensional key points one by one, for example, point-to-point.

In some alternative implementations, please refer to fig. 3, which illustrates a flowchart of another pose evaluation method provided by an embodiment of the present application. As shown in fig. 3, the pose evaluation method may include the following steps 301 to 305.

Step 301, acquiring an image to be processed; the image to be processed comprises an image of a target object;

the implementation process of step 301 and the obtained technical effect may be the same as or similar to step 101 in the embodiment shown in fig. 1, and are not described herein again.

Step 302, according to the image to be processed, acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting time of the image to be processed.

The implementation process of step 302 and the obtained technical effect may be the same as or similar to step 102 in the embodiment shown in fig. 1, and are not described herein again.

Step 303, determining a convex hull of a projection point of the target model projected on the image to be processed under the estimated pose;

the implementation process and the obtained technical effect of step 303 may be the same as or similar to step 103 in the embodiment shown in fig. 1, and are not described herein again.

Step 304, determining a first fit degree between the convex hull of the projection point and the minimum bounding rectangle of the object corresponding to the target object; determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object;

in some application scenes, the minimum bounding rectangle of the object is obtained on line by detecting the image to be processed through a target detection algorithm; or the target object is obtained offline through labeling.

That is, the object minimum bounding rectangle may be detected by a target detection algorithm (e.g., recurrent neural network "RNN"). In other application scenarios, the minimum bounding rectangle of the object can also be obtained by manually labeling the target object.

As previously mentioned, if the estimated pose is the same as the actual pose, then the convex hull of the proxels should coincide with the silhouette polygon of the target object. Whereas the outline polygon of the target object is necessarily contained within the object's minimum bounding rectangle. Thus, the confidence may also be calculated based on the first fit between the convex hull of the proxels and the object's minimum bounding rectangle.

Further, in the related art, the cost of labeling the outline polygon of the target object is high, and a large time cost is also required in detecting the outline polygon. Therefore, in order to save cost, the confidence may also be calculated by using the first fit degree between the convex hull of the projection point and the minimum bounding rectangle of the object corresponding to the target object.

In some application scenarios, if the estimated pose is evaluated only by using the first fit degree, there may be a problem that the convex hull of the projection point is too small. Therefore, in order to further evaluate the accuracy of the estimated pose, the above-described second degree of fitting may be further determined. Here, the reason why the convex hull of the projected point is small may include, for example, that the distance between the camera position and the target object is long when the image to be processed is acquired.

In these application scenarios, the convex hull of the proxels must also be contained within the minimum bounding rectangle of the convex hull. Therefore, the minimum bounding rectangle of the convex hull can be obtained based on the convex hull of the projection point, so that the estimated pose can be evaluated by utilizing the second fit degree between the minimum bounding rectangle of the convex hull and the minimum bounding rectangle of the object. For example, in the vehicle application scene diagram shown in fig. 2, after the convex hull 2 of the projection point is obtained, the minimum bounding rectangle 3 of the convex hull corresponding to the convex hull 2 of the projection point may be obtained. And the vehicle minimum bounding rectangle 4 corresponding to the vehicle can be detected. Thus, the first degree of conformity may include, for example, the degree of conformity between the convex hull 2 of the projected point and the vehicle minimum bounding rectangle 4; the second fitting degree may include, for example, a fitting degree between the convex hull minimum bounding rectangle 3 and the vehicle minimum bounding rectangle 4.

Step 305, calculating the confidence coefficient based on the first fitting degree and the second fitting degree.

After the first and second degrees of fit are determined, the estimated pose may be evaluated based on both.

In some application scenarios, for example, different weights may be assigned to the first fit level and the second fit level, and then confidence levels may be calculated based on the respective weights. For example, a weight coefficient of 0.6 may be assigned to the first degree of attachment and a weight coefficient of 0.4 may be assigned to the second degree of attachment, so that after the first degree of attachment and the second degree of attachment are determined, the first degree of attachment and the second degree of attachment may be respectively calculated with the corresponding weight coefficients to obtain a comprehensive degree of attachment, and the comprehensive degree of attachment may be regarded as a confidence degree calculated based on the first degree of attachment and the second degree of attachment.

Through the steps 301 to 305, the confidence coefficient of the estimated pose can be calculated based on the determined first attaching degree and the second attaching degree, and the confidence coefficient of the estimated pose which is more accurate can be obtained on the basis of effectively saving cost investment.

In some optional implementations, the step 304 may include the following sub-steps:

sub-step 3041, determining a first intersection portion between the convex hull of the proxel and the minimum bounding rectangle of the object;

in some application scenarios, when determining the first fit degree, a first intersection portion between the convex hull of the projection point and the minimum bounding rectangle of the object may be determined. For example, in the schematic diagram of the vehicle application scene shown in fig. 2, the image portion occupied by the convex hull 2 of the projection point can be regarded as the first intersection portion. Here, in other vehicle application scenarios, the area of the convex hull 2 of the projection point may be larger than the area of the vehicle minimum bounding rectangle 4, and in this case, the image portion occupied by the vehicle minimum bounding rectangle 4 may be regarded as the first intersection portion.

Sub-step 3042, characterizing the first fit degree by a first area ratio between the first intersection portion and a convex hull of the projection point;

after the first intersection portion is determined, a first area ratio between the first intersection portion and a convex hull of the projection point may be determined to characterize the first fit degree using the first area ratio. In some application scenarios, the determined first intersection portion may be regarded as a contour portion corresponding to the target object in the image to be processed. The first fit may then be characterized by a first area ratio between the first intersection portion and the convex hull of the projection point.

In some alternative implementations, the first area ratio value may be determined, for example, based on the following equation:wherein, I_OARepresenting said first area ratio, C_PArea of convex hull representing said projection point, C_PAndd denotes an area of the first intersection portion. Therefore, the first area ratio can be obtained by utilizing the formula, so that the first fitting degree is digitalized, and the subsequent evaluation on the estimation pose is facilitated.

Sub-step 3043, determining a second intersection portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle;

in some application scenarios, when determining the second fit degree, a second intersection portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle may be determined. For example, in the schematic diagram of the vehicle application scene shown in fig. 2, the image portion occupied by the minimum bounding rectangle 3 of the convex hull can be regarded as the second intersection portion. Here, in other vehicle application scenarios, the area of the convex-hull minimum bounding rectangle 3 may be larger than the area of the vehicle minimum bounding rectangle 4, and at this time, the image portion occupied by the vehicle minimum bounding rectangle 4 may be regarded as the second intersection portion.

Sub-step 3044, determining a union part between the convex hull minimum bounding rectangle and the object minimum bounding rectangle;

after the second intersection portion is determined, a union portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle may be further determined. For example, in the vehicle application scenario diagram shown in fig. 2, the image portion occupied by the minimum bounding rectangle 4 of the vehicle can be regarded as the union portion. Here, in other vehicle application scenarios, the area of the vehicle minimum bounding rectangle 4 may be smaller than the area of the convex hull minimum bounding rectangle 3, and at this time, the image portion occupied by the convex hull minimum bounding rectangle 3 may be regarded as the union portion.

Sub-step 3045, characterizing the second fit degree by a second area ratio between the second intersection portion and the union portion;

in some application scenarios, after the second intersection part and the union part are obtained, the area of the two parts may be subjected to a quotient so as to obtain the second area ratio. In these application scenarios, in order to avoid the case of inaccurate confidence due to the small overall convex hull such as the projection point, after the second intersection part and the union part are determined, a second fit degree may be characterized based on a second area ratio therebetween to determine whether the minimum bounding rectangle of the convex hull fits the minimum bounding rectangle of the object.

In some alternative implementations, the ratio of the second areas may be determined based on the following equation: i is_OU＝(B_P∩D)/(B_PU.D), wherein I_OUCharacterizing the second area ratio, B_PD denotes the area of the second intersection portion, B_PAnd U.D represents the area of the union part. Therefore, the second area ratio can be obtained by utilizing the formula, so that the second fitting degree is digitized, and the confidence coefficient is conveniently calculated subsequently.

Thus, the above step 305 may include the sub-step 3051: calculating the confidence based on the first area ratio and the second area ratio.

After determining the first area ratio and the second area ratio, a confidence may be calculated based on both. In some application scenarios, the two may be multiplied to obtain the probability that the estimated pose is the actual pose. This probability may be considered the confidence level described above.

In some optional implementations, the sub-step 3051 may include: multiplying the first area ratio and the second area ratioAnd calculating the confidence. For example, the confidence level may be calculated based on the following formula: s ═ I_OA*I_OU(ii) a Wherein S represents the confidence.

That is to say, after the first area ratio and the second area ratio are obtained, the first area ratio and the second area ratio may be multiplied, and the confidence may be determined according to the first fitting degree and the second fitting degree respectively represented by the first fitting degree and the second fitting degree. Therefore, when the confidence coefficient is used for evaluating and estimating the pose, the estimated pose meeting the evaluation condition can be closer to the actual pose.

Please refer to fig. 4, which shows a flowchart of a pose determination method according to an embodiment of the present application. As shown in fig. 4, the pose determination method may include the following steps 401 to 402.

Step 401, obtaining confidence of an estimated pose; the confidence coefficient can be calculated by the pose evaluation method in any of the above implementation manners, which is not described herein again;

and 402, when the confidence coefficient is detected to meet the evaluation condition, determining the estimated pose as the target pose of the target object.

In some application scenarios, after the confidence of the estimated pose is obtained, whether the confidence meets the evaluation condition or not can be detected. The evaluation condition may include, for example, that a linear distance between a plurality of target projection points on a convex hull constituting the projection points and the two-dimensional key point corresponding to each of the target projection points is within 0.1 cm, 0.2 cm, or the like. The two-dimensional key points may be key points corresponding to model points detected in the image to be processed and corresponding to the target projection points.

Further, when it is detected that the confidence degree satisfies the evaluation condition, it can be considered that the current corresponding estimated pose is closer to the actual pose, and then the estimated pose can be determined as the target pose of the target object.

In some optional implementations, the step 402 may include: and when detecting that the numerical value corresponding to the confidence coefficient is larger than a confidence coefficient threshold value, determining the estimated pose as the target pose of the target object.

After the confidence is determined, whether a value corresponding to the confidence is greater than an evaluation threshold value or not can be judged, and when the value corresponding to the confidence is greater than the evaluation threshold value, the estimated pose can be determined as the target pose. The evaluation threshold may include an empirical value or a trial value such as 0.85 or 0.9 that substantially represents that the estimated pose is closer to the actual pose, so as to ensure that the estimated pose satisfying the evaluation condition is closer to the actual pose.

In this embodiment, when it is determined that the confidence is greater than the evaluation threshold, it may be determined that the estimated pose is closer to the actual pose. The estimated pose can thus be determined as the target pose of the target object. The evaluation process is more reasonable, and the reliability of the evaluation result is high.

Referring to fig. 5, a block diagram of a pose evaluation apparatus provided in an embodiment of the present application is shown, where the pose evaluation apparatus may be a module, a program segment, or code on an electronic device. It should be understood that the apparatus corresponds to the above-mentioned embodiment of the method of fig. 1, and can perform various steps related to the embodiment of the method of fig. 1, and the specific functions of the apparatus can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy.

Optionally, the pose determination apparatus includes a first obtaining module 501, a second obtaining module 502, a first determining module 503, and a calculating module 504. The first obtaining module 501 is configured to obtain an image to be processed; the image to be processed comprises an image of a target object; a second obtaining module 502, configured to obtain, according to the image to be processed, a target model corresponding to the target object and an estimated pose of the target object at a shooting time of the image to be processed; a first determining module 503, configured to determine a convex hull of a projection point of the target model projected on the image to be processed in the estimated pose; a calculating module 504, configured to calculate a confidence of the estimated pose based on the convex hull of the projection point and the image of the target object, so as to evaluate the estimated pose according to the confidence.

Optionally, the calculation module 504 is further configured to: determining a contour polygon of the target object in the image to be processed; the outline polygon represents a polygon formed by the outline lines of the target object; and calculating the fit degree between the outline polygon and the convex hull, and determining the fit degree as the confidence coefficient.

Optionally, the calculation module 504 is further configured to: determining a first fit degree between a convex hull of the projection point and a minimum bounding rectangle of the object corresponding to the target object; determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object; calculating the confidence based on the first fit degree and the second fit degree.

Optionally, the calculation module 504 is further configured to: determining a first intersection portion between the convex hull of the proxel and the object minimum bounding rectangle; characterizing the first fit degree by a first area ratio between the first intersection portion and a convex hull of the projection point; and the determining a second fit degree between the minimum circumscribed rectangle of the convex hull corresponding to the convex hull of the projection point and the minimum circumscribed rectangle of the object comprises: determining a second intersection portion between the convex hull minimum bounding rectangle and the object minimum bounding rectangle; determining a union part between the convex hull minimum bounding rectangle and the object minimum bounding rectangle; characterizing the second fit by a second area ratio between the second intersection portion and the union portion; and determining the confidence level based on the first fit level and the second fit level comprises: calculating the confidence based on the first area ratio and the second area ratio.

Optionally, the calculation module 504 is further configured to: and multiplying the first area ratio and the second area ratio to calculate the confidence.

Optionally, the minimum bounding rectangle of the object is obtained on line by detecting the image to be processed through a target detection algorithm; or the target object is obtained offline through labeling.

It should be noted that, for the convenience and brevity of description, the specific working procedure of the above-described apparatus may refer to the corresponding procedure in the foregoing method embodiment, and the description is not repeated herein.

Referring to fig. 6, a block diagram of a pose determination apparatus provided by an embodiment of the present application is shown, where the pose determination apparatus may be a module, a program segment, or code on an electronic device. It should be understood that the apparatus corresponds to the above-mentioned embodiment of the method of fig. 4, and can perform various steps related to the embodiment of the method of fig. 4, and the specific functions of the apparatus can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy.

Optionally, the pose determination apparatus includes a third obtaining module 601 and a second determining module 602; the third obtaining module 601 is configured to obtain a confidence of the estimated pose; the confidence can be obtained by the pose evaluation device shown in fig. 6; a second determining module 602, configured to determine the estimated pose as a target pose of the target object when it is detected that the confidence level satisfies an evaluation condition.

Optionally, the second determining module 602 is further configured to: and when detecting that the numerical value corresponding to the confidence coefficient is larger than a confidence coefficient threshold value, determining the estimated pose as the target pose of the target object.

It should be noted that, for the convenience and brevity of description, the specific working procedure of the above-described apparatus may refer to the corresponding procedure in the foregoing method embodiment, and the description is not repeated herein.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device for executing a pose evaluation method according to an embodiment of the present application, where the electronic device may include: at least one processor 701, e.g., a CPU, at least one communication interface 702, at least one memory 703 and at least one communication bus 704. Wherein a communication bus 704 is used to enable direct, connected communication of these components. In this embodiment, the communication interface 702 of the device in this application is used for performing signaling or data communication with other node devices. The memory 703 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). The memory 703 may optionally also be at least one memory device located remotely from the processor. The memory 703 stores computer readable instructions, which when executed by the processor 701, the electronic device may perform the method processes of fig. 1, for example.

It will be appreciated that the configuration shown in fig. 7 is merely illustrative and that the electronic device may include more or fewer components than shown in fig. 7 or have a different configuration than shown in fig. 7. The components shown in fig. 7 may be implemented in hardware, software, or a combination thereof.

Embodiments of the present application provide a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program may perform the method processes performed by an electronic device in the method embodiment shown in fig. 1.

The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, the method may comprise: acquiring an image to be processed; the image to be processed comprises an image of a target object; according to the image to be processed, acquiring a target model corresponding to the target object and an estimated pose corresponding to the target object at the shooting time of the image to be processed; determining a convex hull of a projection point of the target model projected on the image to be processed under the estimation pose; calculating a confidence of the estimated pose based on the convex hull of the projection points and the image of the target object to evaluate the estimated pose according to the confidence.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.