阅读说明：本技术 用于纹理贴图的方法、装置和存储介质 (Method, apparatus and storage medium for texture mapping ) 是由 程谟方 赵靖 于 2021-09-27 设计创作，主要内容包括：本公开实施例公开了一种用于纹理贴图的方法、装置和存储介质,其中,该方法包括：响应于确定待处理的目标网格中存在待细分片元,将待细分片元细分为至少两个符合预设条件的子片元,并确定待细分片元与至少两个子片元的对应关系,得到细分后的网格,其中,待细分片元为尺寸参数大于预设的尺寸阈值的片元,预设条件为尺寸参数小于或等于尺寸阈值；对细分后的网格进行纹理贴图,得到细分后的网格的纹理贴图结果；基于对应关系,将细分后的网格的纹理贴图结果映射至目标网格,得到目标网格的纹理贴图结果。可以对含有任意大小的片元的网格进行纹理贴图,且不会增加纹理贴图后的网格中片元的数量,从而避免了纹理贴图导致网格的复杂程度增加。(The embodiment of the disclosure discloses a method, a device and a storage medium for texture mapping, wherein the method comprises the following steps: in response to the fact that the target grid to be processed is determined to have the fragments to be subdivided, subdividing the fragments to be subdivided into at least two sub-fragments which meet preset conditions, and determining the corresponding relation between the fragments to be subdivided and the at least two sub-fragments to obtain the subdivided grid, wherein the fragments to be subdivided are the fragments of which the size parameters are larger than a preset size threshold, and the preset conditions are that the size parameters are smaller than or equal to the size threshold; performing texture mapping on the subdivided grid to obtain a texture mapping result of the subdivided grid; and mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid. The method can be used for texture mapping of the grid containing the fragments with any size, and the number of the fragments in the grid after texture mapping is not increased, so that the increase of the complexity of the grid caused by the texture mapping is avoided.)

1. A method for texture mapping, comprising:

in response to the fact that the target grid to be processed is determined to have the fragments to be subdivided, subdividing the fragments to be subdivided into at least two sub-fragments which meet preset conditions, and determining the corresponding relation between the fragments to be subdivided and the sub-fragments to obtain the subdivided grid, wherein the fragments to be subdivided are the fragments of which the size parameters are larger than a preset size threshold, and the preset conditions are that the size parameters are smaller than or equal to the size threshold;

performing texture mapping on the subdivided grid to obtain a texture mapping result of the subdivided grid;

and mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid.

2. The method of claim 1, wherein in response to determining that there are fragments to be subdivided in the target mesh to be processed, subdividing the fragments to be subdivided into at least two sub-fragments meeting a preset condition comprises:

taking the target grid as a current grid;

traversing the current grid;

if the to-be-subdivided fragments exist in the current grid, subdividing the to-be-subdivided fragments into at least two candidate sub-fragments to obtain an updated grid;

taking the updated mesh as a current mesh, and traversing the current mesh again;

if the to-be-subdivided fragments do not exist in the current grid, determining candidate fragments in the current grid as the fragments, and determining the current grid as the subdivided grid.

3. The method of claim 2, wherein the fragment to be subdivided is determined by:

in response to determining that the length of the longest edge of a fragment is greater than a preset length threshold, the fragment is determined to be a fragment to be subdivided.

4. The method of claim 3, wherein subdividing the to-be-subdivided fragment into at least two candidate sub-fragments to obtain an updated mesh comprises:

inserting a new vertex at a preset position of the longest edge of the fragment to be subdivided;

subdividing the fragment to be subdivided into at least two candidate sub-fragments based on the new vertex and the existing old vertex in the fragment to be subdivided to obtain a subdivided current grid;

and in response to determining that each new vertex is a pure composite vertex, determining the subdivided current mesh as an updated mesh, wherein the pure composite vertex represents that the new vertex is only used as a vertex in the fragment containing the new vertex.

5. Method according to one of claims 1 to 4, characterized in that the target grid is obtained by:

extracting a grid surface from the point cloud to be processed, and determining an initial fragment in the grid surface to obtain an initial grid, wherein the point cloud to be processed is the point cloud acquired in a target scene;

and deleting the initial fragments which accord with the preset filtering condition in the initial grid based on a preset grid simplification algorithm to obtain the target grid.

6. The method of claim 5, wherein texture mapping the subdivided mesh to obtain a texture mapping result of the subdivided mesh comprises:

acquiring a texture picture set, wherein the texture picture set comprises texture pictures, and the texture pictures are images collected from the target scene;

determining the relative pose of the texture picture relative to the target grid based on the pose information of the equipment for collecting the texture picture and the pose information of the equipment for collecting the point cloud to be processed;

determining texture pictures corresponding to the fragments in the subdivided grids based on the relative poses of the texture pictures relative to the target grids;

determining texture coordinates of the vertexes in the subdivided mesh based on the geometric coordinates of the vertexes in the subdivided mesh and the texture picture corresponding to the fragments in the subdivided mesh;

and determining a mapping area of the fragments in the subdivided mesh in a texture picture based on the texture coordinates of the vertexes in the subdivided mesh to obtain a texture mapping result of the subdivided mesh.

7. The method of claim 6, wherein mapping the texture mapping result of the subdivided mesh to the target mesh based on the correspondence to obtain the texture mapping result of the target mesh comprises:

determining at least one sub-fragment element set from the texture mapping result of the subdivided grid, wherein each sub-fragment element in each sub-fragment element set corresponds to the same texture picture and is communicated with the same texture picture;

determining a mapping area of each sub-fragment set in the corresponding texture picture based on the texture coordinates of the vertex of each sub-fragment in the sub-fragment set;

determining the sub-fragment set corresponding to the fragment to be subdivided based on the corresponding relation;

determining texture coordinates of the vertex of the to-be-subdivided fragment corresponding to the sub-fragment set based on the mapping area of the sub-fragment set in the corresponding texture picture and the geometric coordinates of the vertex in the to-be-subdivided fragment corresponding to the sub-fragment set;

and determining a mapping area of the to-be-subdivided fragments corresponding to the sub-fragment set in the texture picture based on the texture coordinates of the vertexes of the to-be-subdivided fragments corresponding to the sub-fragment set, so as to obtain a texture mapping result of the to-be-subdivided fragments in the target grid.

8. The method according to claim 6, wherein before mapping the texture mapping result of the subdivided mesh to the target mesh based on the correspondence relationship and obtaining the texture mapping result of the target mesh, the method further comprises:

determining a set of to-be-adjusted fragments from the texture mapping result of the subdivided grid, wherein each fragment in each set of to-be-adjusted fragments corresponds to different texture pictures and is communicated with each other;

and carrying out color adjustment on the mapping area of each fragment in the to-be-adjusted fragment set in the texture picture.

9. An apparatus for texture mapping, comprising:

the fragment subdivision unit is configured to subdivide a to-be-subdivided fragment into at least two sub-fragments meeting a preset condition in response to determining that the to-be-subdivided fragment exists in a to-be-processed target grid, and determine a corresponding relation between the to-be-subdivided fragment and the sub-fragments to obtain a subdivided grid, wherein the to-be-subdivided fragment is a fragment with a size parameter larger than a preset size threshold, and the preset condition is that the size parameter is smaller than or equal to the size threshold;

a texture mapping unit configured to perform texture mapping on the subdivided mesh to obtain a texture mapping result of the subdivided mesh;

and the result mapping unit is configured to map the texture mapping result of the subdivided grid to the target grid based on the corresponding relation, so as to obtain the texture mapping result of the target grid.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method for texture mapping according to any one of the claims 1 to 8.

Technical Field

The present disclosure relates to the field of computer vision, and more particularly, to a method, an apparatus, and a storage medium for texture mapping.

Background

Texture mapping is a technique that uses images, functions, or other data sources to change the appearance of a model (e.g., a mesh generated based on a point cloud) surface. By combining the image and the model surface in this way, the model can be made to look more realistic.

When the model is subjected to texture mapping, an overlarge fragment in the model cannot be covered by any view angle, and further the fragment cannot be subjected to texture mapping.

In the related art, an oversized fragment is usually subdivided, and then texture mapping is performed on the subdivided fragment, so that texture mapping on a model is realized, and the number and complexity of the fragments in the model are increased.

Disclosure of Invention

The embodiment of the disclosure provides a method, a device and a storage medium, so as to solve the problem that in the related art, after texture mapping is performed on a model after fragment subdivision, the complexity of the model is increased.

In one aspect of the disclosed embodiments, there is provided a method for texture mapping, including: in response to the fact that the target grid to be processed is determined to have the fragments to be subdivided, subdividing the fragments to be subdivided into at least two sub-fragments which meet preset conditions, and determining the corresponding relation between the fragments to be subdivided and the sub-fragments to obtain the subdivided grid, wherein the fragments to be subdivided are the fragments of which the size parameters are larger than a preset size threshold, and the preset conditions are that the size parameters are smaller than or equal to the size threshold; performing texture mapping on the subdivided grid to obtain a texture mapping result of the subdivided grid; and mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid.

In some embodiments, in response to determining that there are fragments to be subdivided in the target mesh to be processed, subdividing the fragments to be subdivided into at least two sub-fragments meeting a preset condition, the method includes: taking the target grid as a current grid; traversing the current grid; if the current grid has the fragment to be subdivided, subdividing the fragment to be subdivided into at least two candidate sub-fragments to obtain an updated grid; taking the updated grid as the current grid, and traversing the current grid again; and if the current grid does not have the fragment to be subdivided, determining the candidate fragment in the current grid as the fragment, and determining the current grid as the subdivided grid.

In some embodiments, the fragment to be subdivided is determined via: in response to determining that the length of the longest edge of a fragment is greater than a preset length threshold, the fragment is determined to be a fragment to be subdivided.

In some embodiments, subdividing the to-be-subdivided piece into at least two candidate sub-pieces to obtain an updated mesh, includes: inserting a new vertex at a preset position of the longest edge of the fragment to be subdivided; subdividing the fragment to be subdivided into at least two candidate sub-fragments based on the new vertex and the existing old vertex in the fragment to be subdivided to obtain a subdivided current grid; and in response to determining that each new vertex is a pure composite vertex, determining the subdivided current mesh as an updated mesh.

In some embodiments, the target mesh is obtained via the steps of: extracting a grid surface from the point cloud to be processed, determining an initial fragment in the grid surface to obtain an initial grid, wherein the point cloud to be processed is the point cloud acquired in a target scene; and deleting the initial fragments which accord with the preset filtering condition in the initial grid based on a preset grid simplification algorithm to obtain the target grid.

In some embodiments, texture mapping the subdivided mesh to obtain a texture mapping result of the subdivided mesh includes: acquiring a texture picture set, wherein the texture picture set comprises texture pictures, and the texture pictures are images acquired from a target scene; determining the relative pose of the texture picture relative to the target grid based on the pose information of the equipment for acquiring the texture picture and the pose information of the equipment for acquiring the point cloud to be processed; determining texture pictures corresponding to the fragments in the subdivided grids based on the relative poses of the texture pictures relative to the target grids; determining texture coordinates of the vertexes in the subdivided mesh based on the geometric coordinates of the vertexes in the subdivided mesh and the texture picture corresponding to the fragments in the subdivided mesh; and determining a mapping area of the fragments in the subdivided mesh in the texture picture based on the texture coordinates of the vertexes in the subdivided mesh to obtain a texture mapping result of the subdivided mesh.

In some embodiments, mapping the texture mapping result of the subdivided mesh to the target mesh based on the correspondence, to obtain the texture mapping result of the target mesh, includes: determining at least one sub-fragment element set from the texture mapping result of the subdivided grid, wherein each sub-fragment element in each sub-fragment element set corresponds to the same texture picture and is communicated with the same texture picture; determining a mapping area of each sub-fragment set in the corresponding texture picture based on the texture coordinates of the vertex of each sub-fragment in the sub-fragment set; determining the fragments to be subdivided corresponding to the sub-fragment set based on the corresponding relation; determining texture coordinates of the vertexes of the to-be-subdivided fragments corresponding to the sub-fragment sets on the basis of the mapping areas of the sub-fragment sets in the corresponding texture pictures and the geometrical coordinates of the vertexes of the to-be-subdivided fragments corresponding to the sub-fragment sets; and determining a mapping area of the to-be-subdivided fragments corresponding to the sub-fragment set in the texture picture based on the texture coordinates of the vertexes of the to-be-subdivided fragments corresponding to the sub-fragment set, and obtaining a texture mapping result of the to-be-subdivided fragments.

In some embodiments, the texture mapping result of the subdivided mesh is mapped to the target mesh based on the correspondence, so as to obtain the texture mapping result of the target mesh, and the method further includes: determining a set of to-be-adjusted fragments from the texture mapping result of the subdivided grid, wherein each fragment in each set of to-be-adjusted fragments corresponds to different texture pictures and is communicated with each other; and carrying out color adjustment on the mapping area of each fragment in the to-be-adjusted fragment set in the texture picture.

In yet another aspect of the disclosed embodiments, there is provided an apparatus for texture mapping, including: the fragment subdivision unit is configured to subdivide the to-be-subdivided fragments into at least two sub-fragments meeting preset conditions in response to the fact that the to-be-subdivided fragments exist in the to-be-processed target grid, and determine the corresponding relation between the to-be-subdivided fragments and the sub-fragments to obtain a subdivided grid, wherein the to-be-subdivided fragments are the fragments with the size parameters larger than a preset size threshold, and the preset conditions are that the size parameters are smaller than or equal to the size threshold; a texture mapping unit configured to perform texture mapping on the subdivided mesh to obtain a texture mapping result of the subdivided mesh; and the result mapping unit is configured to map the texture mapping result of the subdivided grid to the target grid based on the corresponding relation, so as to obtain the texture mapping result of the target grid.

In yet another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor, implements the method for texture mapping in any of the above embodiments.

The method for texture mapping comprises the steps that firstly, fragments to be subdivided, of which the size parameters are larger than a preset size threshold value, in a target grid are at least two sub-fragments meeting preset conditions, the corresponding relation between the fragments to be subdivided and the sub-fragments is determined, and a subdivided grid value is obtained; then texture mapping is carried out on the subdivided grids to obtain texture mapping results of the subdivided grids; and then mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid. The method can be used for texture mapping of the grid containing the fragments with any size, and the number of the fragments in the grid after texture mapping cannot be increased, so that the increase of the complexity of the grid caused by the texture mapping is avoided, and extra operation pressure is not brought to the processing processes of storage, transmission, rendering and the like of the subsequent grid. The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.

The present disclosure may be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:

FIG. 1 is an exemplary system architecture diagram in which some embodiments of the present disclosure may be applied;

FIG. 2 is a flow diagram of one embodiment of a method for texture mapping of the present disclosure;

FIG. 3 is a mapping result of the subdivided mesh in the implementation shown in FIG. 2;

FIG. 4 is a mapping result of the target grid in the embodiment shown in FIG. 2;

FIG. 5 is a flow chart illustrating the subdivision of a fragment to be subdivided into at least two sub-fragments in the embodiment shown in FIG. 2;

FIG. 6 is a flow chart of determining an updated mesh in the embodiment shown in FIG. 3;

FIG. 7 is a flow diagram of yet another embodiment of a method for texture mapping of the present disclosure;

FIG. 8 is a flow diagram of yet another embodiment of a method for texture mapping of the present disclosure;

FIG. 9 is a schematic diagram of an embodiment of an apparatus for texture mapping according to the present disclosure;

fig. 10 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure.

Detailed Description

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

In the course of implementing the present disclosure, the inventors found that, when performing texture mapping on a model containing an oversized fragment in the related art, the oversized fragment is usually subdivided into smaller fragments, and then the subdivided model is subjected to texture mapping, which is used as a texture mapping result of the model. Therefore, the number of fragments in the texture mapped model is increased, and further the texture mapped model has more redundant data, which brings extra operation pressure to the subsequent processing processes of storage, transmission, rendering and the like.

Referring to fig. 1, fig. 1 illustrates an exemplary system architecture 100 of a method for texture mapping or an apparatus for texture mapping of an embodiment of the present disclosure.

As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to send or receive data or the like, for example, the target mesh to be processed may be sent to the server, and texture mapping results of the target mesh may also be received from the server.

The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they may be electronic devices with communication functions, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.

The server 105 may be a server providing a texture mapping service, e.g. a background application server performing texture mapping on the target mesh sent by the terminal devices 101, 102, 103. The background application server may perform texture mapping according to the received target mesh, and feed back a processing result (e.g., a texture mapping result) to the terminal device.

The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules, for example, to provide distributed services, or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the method for texture mapping provided by the embodiments of the present disclosure may be executed by the terminal devices 101, 102, 103, or may be executed by the server 105. Accordingly, the means for texture mapping may be provided in the terminal devices 101, 102, 103, or in the server 105. And is not particularly limited herein.

With continued reference to FIG. 2, FIG. 2 illustrates a flow diagram for one embodiment of the present disclosure for texture mapping. As shown in fig. 2, the process includes steps 210 to 230.

Step 210, in response to determining that the target grid to be processed has the fragment to be subdivided, subdividing the fragment to be subdivided into at least two sub-fragments meeting preset conditions, and determining a corresponding relationship between the fragment to be subdivided and the sub-fragments to obtain the subdivided grid.

The to-be-subdivided fragment is a fragment whose size parameter is greater than a preset size threshold, the preset condition is that the size parameter is less than or equal to the size threshold, and the size parameter may be, for example, the area of the fragment or the side length of the fragment.

In this embodiment, the correspondence between the to-be-subdivided piece element and the sub-piece element may represent the dependency relationship between the to-be-subdivided piece element and the sub-piece element, for example, the to-be-subdivided piece element a is subdivided into the sub-piece elements a1, a2, and A3, and then the correspondence between the to-be-subdivided piece element a and the sub-piece elements a1, a2, and A3 may be determined and stored in a pre-constructed correspondence list, in which each piece element may be represented by vertex information (for example, may include the geometric coordinates of a vertex).

In this embodiment, the target mesh is mesh (mesh) data constructed based on point cloud, for example, a user may collect point cloud of a scene through an RGB-D camera or a point cloud radar, and then apply a mesh generation algorithm to the collected point cloud to extract a mesh surface from the point cloud, so as to obtain mesh data including fragments.

In one specific example, the execution agent may be the server 105 shown in fig. 1, which may be preloaded with a mesh subdivision algorithm, such as a tessellation algorithm (loop subdivision). The user may send the target mesh to be processed to the server 105 through the terminal device. Then, the server 105 traverses each fragment in the target grid, determines the fragment with an area larger than a preset area threshold from the target grid, and determines the fragment as a fragment to be subdivided. Then, the server 105 executes a mesh subdivision algorithm with the fragment to be subdivided as an object, inserts a new vertex on each edge of the fragment to be subdivided, subdivides each fragment to be subdivided into at least two sub-fragments of which the areas are not larger than a preset area threshold value based on the new vertex and an existing old vertex in the fragment to be subdivided, and obtains a subdivided mesh, and meanwhile, the server 105 may store the correspondence between each fragment to be subdivided and at least two sub-fragments into which the fragment to be subdivided into a correspondence list constructed in advance.

It should be noted that the shape of the slice element in the present disclosure may be a triangle or other polygons, which is not limited in the present disclosure.

Step 220, performing texture mapping on the subdivided grid to obtain a texture mapping result of the subdivided grid.

In this embodiment, the fragments included in the subdivided grid obtained in step 210 include two types, one is an initial fragment whose size parameter is smaller than or equal to the size threshold in the target grid, and the other is a sub-fragment obtained by subdividing the initial fragment whose size parameter is greater than the size threshold. The execution main body can determine texture coordinates of each vertex in the subdivided grid by executing a preset texture mapping algorithm on the subdivided grid, and then, the vertex shader is used for shading the fragment to generate a mapping result of the subdivided grid.

As an example, the execution subject may input the subdivided mesh and a predetermined two-dimensional picture into OpenGL (Open Graphics Library), and determine a mapping relationship between each slice in the subdivided mesh and the two-dimensional picture according to a predetermined mapping function in OpenGL, so as to obtain a mapping result of the subdivided mesh.

With further reference to fig. 3, fig. 3 shows the mapping result of the subdivided mesh in the present embodiment.

And step 230, mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid.

In this embodiment, the execution main body may determine, according to the correspondence between the fragment to be subdivided and the sub-fragment, the dependency between the sub-fragment in the subdivided mesh and the fragment to be subdivided in the target mesh, and may further determine, according to the texture mapping result of the sub-fragment, the texture mapping result of the fragment to be subdivided.

With further reference to fig. 4, fig. 4 shows the mapping result of the target grid in the present embodiment.

As an example, the execution subject may use a barycentric method to map the texture mapping result of each fragment in the subdivided grid to each fragment in the target grid through affine transformation, so as to obtain the texture mapping result of the target grid.

As can be seen from fig. 2, 3 and 4, in the method for texture mapping of the present disclosure, first, the to-be-subdivided fragments whose size parameters are greater than the preset size threshold in the target mesh are at least two sub-fragments that meet the preset condition, and the correspondence between the to-be-subdivided fragments and the sub-fragments is determined, so as to obtain the subdivided mesh value; then texture mapping is carried out on the subdivided grids to obtain texture mapping results of the subdivided grids; and then mapping the texture mapping result of the subdivided grid to the target grid based on the corresponding relation to obtain the texture mapping result of the target grid. The method can be used for texture mapping of the grid containing the fragments with any size, and the number of the fragments in the grid after texture mapping cannot be increased, so that the increase of the complexity of the grid caused by the texture mapping is avoided, and extra operation pressure is not brought to the processing processes of storage, transmission, rendering and the like of the subsequent grid.

Referring next to fig. 5, as shown in fig. 5, in some optional implementations of this embodiment, step 210 may further include step 510 to step 550.

And step 510, taking the target grid as the current grid.

Step 520, traverse the current grid.

And 530, if the fragment to be subdivided exists in the current grid, subdividing the fragment to be subdivided into at least two candidate sub-fragments to obtain an updated grid.

In this implementation manner, the execution subject may subdivide the fragments whose size parameters are greater than the preset threshold into at least two candidate sub-fragments with smaller sizes, and then update the fragments in the target grid according to the subdivision result, so as to obtain an updated grid.

And 540, taking the updated grid as the current grid, and traversing the current grid again.

It is understood that it is still possible that the size parameter of the subdivided candidate sub-fragments is larger than the size threshold, and by traversing the current mesh again, the executing entity may subdivide the candidate sub-fragments whose size parameter is still larger than the size threshold until the size parameter of all the fragments (including the initial fragment and the subdivided candidate sub-fragments) in the current mesh is smaller than or equal to the size threshold.

Step 550, if there is no fragment to be subdivided in the current grid, determining the candidate fragment in the current grid as a fragment, and determining the current grid as a subdivided grid.

As can be seen from fig. 5, the process shown in fig. 5 embodies that the mesh to be subdivided in the target mesh is subdivided by circularly traversing the current mesh, so that it is ensured that no fragment with an excessively large size exists in the subdivided mesh.

In an alternative example of implementation shown in fig. 5, the fragment to be subdivided is determined via: in response to determining that the length of the longest edge of a fragment is greater than a preset length threshold, the fragment is determined to be a fragment to be subdivided. In this example, the length of the longest side of a fragment may be taken as the size parameter of the fragment, and accordingly, the length threshold of the longest side may be taken as the size threshold. So, carry out the main part and only need follow each film element in the summit ergodic current net, determine the length on the longest side of film element, then the length and the length threshold value on longest side of contrast can confirm whether the film element is for waiting to subdivide the film element, can reduce the operand among the subdivision process, improve subdivision efficiency.

With further reference to fig. 6, as shown in fig. 6, on the basis of this example, the step 530 may further include steps 610 to 630.

And 610, inserting a new vertex at a preset position of the longest edge of the fragment to be subdivided.

As an example, the preset position may be determined according to a preset ratio or a preset distance.

Preferably, the preset position may be a midpoint of the longest side.

And step 620, subdividing the fragment to be subdivided into at least two candidate sub-fragments based on the new vertex and the existing old vertex in the fragment to be subdivided to obtain the subdivided current grid.

Taking a triangle fragment as an example, the execution main body may connect the old vertex and the new vertex, which are opposite to each other at the longest edge, and the obtained connecting line divides the fragment to be subdivided into two candidate sub-fragments.

Step 630, in response to determining that each new vertex is a pure composite vertex, determining the subdivided current mesh as an updated mesh.

In this embodiment, a simple composite (simple complex) vertex characterizes the inserted new vertex as a vertex of a fragment only. As an example, if the new vertex is a vertex in two adjacent fragments, the new vertex is a pure compound vertex; and if the new vertex is not taken as the vertex in one fragment, the new vertex is a non-simple compound type.

As can be seen from fig. 6, the flow shown in fig. 6 embodies the step of subdividing the fragment to be subdivided by inserting the new vertex at the longest edge, so that the subdivision efficiency can be further improved.

Referring next to fig. 7, fig. 7 illustrates a flowchart of yet another embodiment of the method for texture mapping of the present disclosure, as shown in fig. 7, the flowchart includes steps 710 to 780.

Step 710, extracting a grid surface from the point cloud to be processed, and determining an initial fragment in the grid surface to obtain an initial grid.

In this embodiment, the point cloud to be processed is a point cloud acquired in a target scene.

In practice, a user can acquire point clouds of a target scene through an RGB-D camera, a binocular camera or a point cloud radar, and then execute a preset grid generation algorithm on the acquired point clouds to obtain a target grid to be processed. As an example, for a point cloud with a normal vector, an executing entity may extract an implicit surface of the point cloud by using a poisson reconstruction algorithm, and then extract a mesh surface according to a matching cube (matching cube) algorithm or a derivative algorithm thereof, so as to obtain a target mesh. For point clouds without normal vectors, the execution main body can adopt a Delaynay algorithm to subdivide the point clouds, and then adopts a graph cut algorithm to extract a grid surface to obtain a target grid.

And 720, deleting the initial fragments which meet the preset filtering condition in the initial grid based on a preset grid simplification algorithm to obtain the target grid.

Generally, grid data obtained through a grid generation algorithm has more redundant data, which results in an excessive amount of computation in the subsequent processing. Thus, there is a need to prune the redundant data in the initial mesh.

As an example, the executing body may adopt a vertex clustering (vertex clustering) algorithm, an edge collapsing (edge collapse) algorithm, or a progressive mesh (progressive mesh) algorithm to simplify the initial mesh, delete the initial fragments that meet the preset filtering condition, and retain the geometric characteristics and topological relationships of the fragments in the initial mesh to obtain the target mesh.

Step 730, in response to determining that the target grid to be processed has the fragment to be subdivided, subdividing the fragment to be subdivided into at least two sub-fragments meeting preset conditions, and determining the corresponding relationship between the fragment to be subdivided and the sub-fragments to obtain the subdivided grid.

This step corresponds to the step 210, and is not described herein again.

And step 740, acquiring a texture picture set.

In this embodiment, the texture picture set includes texture pictures, and the texture pictures are images collected from a target scene.

In the implementation, the texture picture and the point cloud to be processed are both collected from the target scene, and the texture picture is mapped to the target grid, so that the target grid can more accurately and truly present the target scene.

And 750, determining the relative pose of the texture picture relative to the target grid based on the pose information of the equipment for acquiring the texture picture and the pose information of the equipment for acquiring the point cloud to be processed.

In this embodiment, the relative pose of the texture picture with respect to the target mesh is used to characterize the relative perspective of the texture picture and the target mesh.

As an example, the execution subject may determine, through coordinate transformation, pose information of the image acquisition device in a world coordinate system according to pixel coordinates of a texture image and a device coordinate system of a device acquiring the texture image; the executing body can determine the pose information of the point cloud acquisition equipment in the world coordinate system according to the position information of each point in the point cloud to be processed and the equipment coordinate system of the point cloud acquisition equipment. Then, the difference value of the pose information of the image acquisition equipment and the pose information of the point cloud acquisition equipment can be used as the relative pose of the texture image relative to the target grid.

And 760, determining the texture picture corresponding to the fragment in the subdivided grid based on the relative pose of the texture picture relative to the target grid.

Step 770, determining texture coordinates of the vertices in the subdivided mesh based on the texture pictures corresponding to the geometric coordinates of the vertices in the subdivided mesh and the fragments in the subdivided mesh.

In this embodiment, the executing entity may map the vertex of each fragment in the subdivided mesh to the corresponding texture picture by using a preset mapping function, and determine the texture coordinate of each vertex.

As an example, the executing entity may store the texture picture determined in step 550 into a texture space, and then map each vertex to the texture space based on the geometric coordinates of each vertex of the corresponding fragment of the texture picture by using a preset mapping function, so as to determine the texture coordinates of each vertex.

Step 780, determining a mapping region of the fragment in the subdivided mesh in the texture picture based on the texture coordinate of the vertex in the subdivided mesh, and obtaining a texture mapping result of the subdivided mesh.

As can be seen from fig. 7, in the embodiment shown in fig. 7, the execution subject may determine the degree of view matching between the fragments in the subdivided grid and the texture picture through the relative pose of the texture picture with respect to the target grid, so as to select the most suitable texture picture for each fragment for mapping, which may make the texture mapping result of the target grid closer to the real scene and improve the accuracy of the texture mapping.

Referring next to fig. 8, the method may further include steps 810 to 850 as follows based on the embodiment shown in fig. 7.

Step 810, determining at least one sub-patch element set from the texture mapping result of the subdivided grid.

In this embodiment, each sub-slice in each sub-slice set corresponds to the same texture picture and is connected.

It can be understood that a plurality of sub-slices obtained by subdividing one slice to be subdivided are in the same plane and are connected, and the viewing angles thereof are the same, then the texture picture determined for each sub-slice in step 760 is also the same, and the execution subject may determine the sub-slices in the subdivided table.

And step 820, determining a mapping area of the sub-fragment set in the corresponding texture picture based on the texture coordinates of the vertex of each sub-fragment in the sub-fragment set.

And 830, determining the fragment to be subdivided corresponding to the sub-fragment set based on the corresponding relation.

And 840, determining texture coordinates of the vertex of the to-be-subdivided fragment corresponding to the sub-fragment set based on the mapping area of the sub-fragment set in the corresponding texture picture and the geometric coordinates of the vertex of the to-be-subdivided fragment corresponding to the sub-fragment set.

In a specific example, the execution subject may respectively extract mapping regions corresponding to a plurality of sub-patch sets from a texture picture, and arrange the mapping regions as texture patches (texture patches) into a texture map set to be mapped. And then, mapping the vertex of the to-be-subdivided fragment to a texture atlas to be mapped by adopting a preset mapping function based on the geometric coordinate of the vertex of the to-be-subdivided fragment corresponding to each sub-fragment set, and determining the texture coordinate of the vertex of each to-be-subdivided fragment.

And 850, determining a mapping area of the to-be-subdivided fragments corresponding to the sub-fragment set in the texture picture based on the texture coordinates of the vertexes of the to-be-subdivided fragments corresponding to the sub-fragment set, and obtaining a texture mapping result of the to-be-subdivided fragments.

As can be seen from fig. 8, in the embodiment shown in fig. 8, the sub-fragment set corresponding to each fragment to be subdivided can be identified from the mapping result of the subdivided mesh, and the texture mapping result of the sub-fragment set is mapped to the fragment to be subdivided to obtain the texture mapping result of the fragment to be subdivided, and the texture mapping result of the subdivided mesh can be more accurately mapped to the target mesh, so as to further improve the accuracy of texture mapping.

In some optional implementations of this embodiment, before step 810, the method may further include: determining a set of to-be-adjusted fragments from the texture mapping result of the subdivided grid, wherein each fragment in each set of to-be-adjusted fragments corresponds to different texture pictures and is communicated with each other; and carrying out color adjustment on the mapping area of each fragment in the to-be-adjusted fragment set in the texture picture.

In this implementation, the execution main body may improve continuity of the texture mapping result by performing color adjustment on the texture mapping result of the to-be-adjusted primitive set.

Referring next to fig. 9, fig. 9 shows a schematic structural diagram of an embodiment of the apparatus for texture mapping of the present disclosure, as shown in fig. 9, the apparatus comprising: the fragment subdividing unit 910 is configured to, in response to determining that a to-be-subdivided fragment exists in a target grid to be processed, subdivide the to-be-subdivided fragment into at least two sub-fragments meeting a preset condition, and determine a correspondence between the to-be-subdivided fragment and the sub-fragments to obtain a subdivided grid, where the to-be-subdivided fragment is a fragment whose size parameter is greater than a preset size threshold, and the preset condition is that the size parameter is less than or equal to the size threshold; a texture mapping unit 920 configured to perform texture mapping on the subdivided mesh to obtain a texture mapping result of the subdivided mesh; and a result mapping unit 930 configured to map the texture mapping result of the subdivided mesh to the target mesh based on the correspondence, so as to obtain a texture mapping result of the target mesh.

In this embodiment, the fragment subdivision unit 910 includes: a mesh determination module configured to take the target mesh as a current mesh; a traversal module configured to traverse a current mesh; the subdivision module is configured to subdivide the to-be-subdivided fragments into at least two candidate sub-fragments if the to-be-subdivided fragments exist in the current grid to obtain an updated grid; a mesh update module configured to take the updated mesh as a current mesh and traverse the current mesh again; and the determining module is configured to determine the candidate sub-fragments in the current grid as the sub-fragments if the current grid does not have the fragments to be subdivided, and determine the current grid as the subdivided grid.

In this embodiment, the subdividing module is further configured to: in response to determining that the length of the longest edge of a fragment is greater than a preset length threshold, the fragment is determined to be a fragment to be subdivided.

In this embodiment, the subdividing module further includes: the vertex inserting sub-module is configured to insert a new vertex at a preset position of the longest edge of the fragment to be subdivided; the fragment subdivision submodule is configured to subdivide the fragment to be subdivided into at least two candidate sub-fragments based on the new vertex and the existing old vertex in the fragment to be subdivided, so as to obtain a subdivided current grid; a result determination submodule configured to determine the subdivided current mesh as an updated mesh in response to determining that each new vertex is a purely composite vertex.

In this embodiment, the apparatus further includes a target mesh generation unit, and the target mesh generation unit includes: the system comprises a grid extraction module, a target scene acquisition module and a data processing module, wherein the grid extraction module is configured to extract a grid surface from a point cloud to be processed, and determine an initial fragment in the grid surface to obtain an initial grid; and the grid simplifying module is configured to delete the initial fragments meeting the preset filtering condition in the initial grid based on a preset grid simplifying algorithm to obtain the target grid.

In this embodiment, the texture mapping unit 920 further includes: the image acquisition module is configured to acquire a texture picture set, wherein the texture picture set comprises texture pictures, and the texture pictures are images acquired from a target scene; a relative pose determination module configured to determine a relative pose of the texture picture with respect to the target mesh based on pose information of a device that acquired the texture picture and pose information of a device that acquired the point cloud to be processed; the picture determining module is configured to determine a texture picture corresponding to the fragment in the subdivided grid based on the relative pose of the texture picture relative to the target grid; a first coordinate determination module configured to determine texture coordinates of vertices in the subdivided mesh based on the texture picture in which the geometric coordinates of the vertices in the subdivided mesh correspond to the fragments in the subdivided mesh; and the first coordinate mapping module is configured to determine a mapping area of the fragments in the subdivided grid in the texture picture based on the texture coordinates of the vertexes in the subdivided grid, so as to obtain a texture mapping result of the subdivided grid.

In this embodiment, the result mapping unit 930 further includes: a sub-fragment extraction module configured to determine at least one sub-fragment set from texture mapping results of the subdivided mesh, each sub-fragment in each sub-fragment set corresponding to and communicating with the same texture picture; the mapping region determining module is configured to determine a mapping region of each sub-fragment set in the corresponding texture picture based on texture coordinates of a vertex of each sub-fragment in the sub-fragment set; the to-be-subdivided fragment determining module is configured to determine the to-be-subdivided fragments corresponding to the sub-fragment sets based on the corresponding relation; the second coordinate determination module is configured to determine texture coordinates of the vertex of the to-be-subdivided fragment corresponding to the sub-fragment set based on the mapping area of the sub-fragment set in the corresponding texture picture and the geometric coordinates of the vertex in the to-be-subdivided fragment corresponding to the sub-fragment set; and the second coordinate mapping module is configured to determine a mapping area of the to-be-subdivided fragment corresponding to the sub-fragment set in the texture picture based on the texture coordinate of the vertex of the to-be-subdivided fragment corresponding to the sub-fragment set, so as to obtain a texture mapping result of the to-be-subdivided fragment in the target grid.

In this embodiment, the apparatus further includes a color adjustment unit, the color adjustment unit including: the target determining module is configured to determine a set of to-be-adjusted fragments from texture mapping results of the subdivided grids, and each fragment in the set of to-be-adjusted fragments corresponds to different texture pictures and is communicated with the different texture pictures; and the adjusting module is configured to perform color adjustment on the mapping area of each fragment in the to-be-adjusted fragment set in the texture picture.

Fig. 10 is a schematic structural diagram of an embodiment of an application of the electronic device of the present disclosure. Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 8. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.

As shown in fig. 10, the electronic device includes one or more processors and memory.

The processor may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

The memory may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by a processor to implement the methods for texture mapping of the various embodiments of the present disclosure described above and/or other desired functions.

In one example, the electronic device may further include: an input device and an output device, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device may also include, for example, a keyboard, a mouse, and the like.

The output device may output various information including the determined distance information, direction information, and the like to the outside. The output devices may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device relevant to the present disclosure are shown in fig. 10, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device may include any other suitable components, depending on the particular application.

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method for texture mapping according to various embodiments of the present disclosure described in the above-mentioned part of the specification.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in the method for texture mapping according to various embodiments of the present disclosure described in the above section of the present specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.