阅读说明：本技术 在增强现实中渲染虚拟物体的方法及装置 (The method and device of dummy object is rendered in augmented reality ) 是由 白雪莲 于 2019-08-22 设计创作，主要内容包括：本发明提供了一种在增强现实中渲染虚拟物体的方法及装置,所述方法包括以下步骤：S1：获取现实场景的图像数据；S2：根据虚拟场景中的虚拟物体的相关信息从获取的现实场景的图像数据中识别出参考物体的图像数据；S3：使用人工智能算法将参考物体的图像数据的光照数据迁移至虚拟物体的图像数据中以得出目标虚拟物体。本发明基于人工智能算法将现实场景中的参考物体的光照迁移至虚拟场景中的虚拟物体上,使虚拟物体与现实场景更真实的融合。(The present invention provides a kind of in augmented reality renders the method and device of dummy object, the described method comprises the following steps: S1: obtaining the image data of reality scene；S2: the image data of Reference is identified from the image data of the reality scene of acquisition according to the relevant information of the dummy object in virtual scene；S3: the photometric data of the image data of Reference is migrated into the image data of dummy object to obtain destination virtual object using intelligent algorithm.The present invention is based on intelligent algorithms to migrate the illumination of the Reference in reality scene on the dummy object into virtual scene, merges dummy object more really with reality scene.)

1. a kind of method for rendering dummy object in augmented reality, which is characterized in that the described method comprises the following steps:

S1: the image data of reality scene is obtained；

S2: ginseng is identified from the image data of the reality scene of acquisition according to the relevant information of the dummy object in virtual scene Examine the image data of object；

S3: the photometric data of the image data of Reference is migrated to the image data of dummy object using intelligent algorithm In to obtain destination virtual object.

2. the method as described in claim 1, which is characterized in that the relevant information of the dummy object includes the material of dummy object The location information of matter parameter and/or dummy object in virtual scene.

3. method according to claim 2, which is characterized in that the step S2 includes:

It is identified from the image data of the reality scene of acquisition according to the material parameters of dummy object and dummy object material class As generic object image data, or according to location information of the dummy object in virtual scene from the real field of acquisition The image data with the object of dummy object being positioned adjacent to is identified in the image data of scape.

4. method as claimed in claim 3, which is characterized in that the step S3 includes:

The photometric data of the image data of the generic object similar with dummy object material is migrated to the image of dummy object In data, or the photometric data of the image data of the object being positioned adjacent to dummy object migrated to the figure of dummy object As in data.

5. method as claimed in claim 4, which is characterized in that by the image of the generic object similar with dummy object material The step that the photometric data of data is migrated into the image data of dummy object includes:

The 3D model of dummy object in virtual scene is tentatively rendered；

The photometric data of generic object is migrated on the dummy object to after tentatively rendering to obtain based on intelligent algorithm The image data of dummy object with illumination；

Secondary rendering is carried out to obtain destination virtual object to the image data of the dummy object with illumination.

6. method as claimed in claim 4, which is characterized in that by the image data for the object being positioned adjacent to dummy object The step that photometric data is migrated into the image data of dummy object includes:

The 3D model of dummy object in virtual scene is tentatively rendered；

The photometric data for the object being positioned adjacent to dummy object and shadow data are migrated to preliminary based on intelligent algorithm To obtain destination virtual object on dummy object after rendering.

7. such as method described in claim 5 or 6, which is characterized in that the dummy object in virtual scene carries out preliminary The step of rendering includes:

According to the attitude data of active user come use environment light rendering dummy object to obtain the preliminary wash with watercolours of user's current visual angle The image data of the dummy object of dye.

8. method as claimed in claim 5, which is characterized in that the image data to the dummy object with illumination Secondary rendering is carried out to obtain includes: the step of destination virtual object

Shadow recognition is carried out to the image data of the dummy object with illumination to obtain the direction of illumination of dummy object, and foundation The plan-position that the direction of illumination and dummy object of the dummy object of acquisition are placed renders the shadow image of dummy object.

9. a kind of device for rendering dummy object in augmented reality, which is characterized in that described device includes:

Contextual data module is configured as obtaining the image data of reality scene；

Object identification module is configured as according to the relevant information of the dummy object in virtual scene from the reality scene of acquisition The image data of Reference is identified in image data；

Data processing module, be configured with intelligent algorithm by the photometric data of the image data of Reference migrate to To obtain destination virtual object in the image data of dummy object.

10. device as claimed in claim 9, which is characterized in that the relevant information of the dummy object includes dummy object The location information of material parameters and/or dummy object in virtual scene.

11. device as claimed in claim 10, which is characterized in that the object identification module is configured as:

It is identified from the image data of the reality scene of acquisition according to the material parameters of dummy object and dummy object material class As generic object image data, or according to location information of the dummy object in virtual scene from the real field of acquisition The image data with the object of dummy object being positioned adjacent to is identified in the image data of scape.

12. device as claimed in claim 11, which is characterized in that the data processing module is configured as:

The photometric data of the image data of the generic object similar with dummy object material is migrated to the image of dummy object In data, or the photometric data of the image data of the object being positioned adjacent to dummy object migrated to the figure of dummy object As in data.

13. device as claimed in claim 12, which is characterized in that the data processing module pair is similar with dummy object material Generic object image data carry out illumination migration when, be configured as:

The 3D model of dummy object in virtual scene is tentatively rendered；

The photometric data of generic object is migrated on the dummy object to after tentatively rendering to obtain based on intelligent algorithm The image data of dummy object with illumination；

Secondary rendering is carried out to obtain destination virtual object to the image data of the dummy object with illumination.

14. device as claimed in claim 12, which is characterized in that the data processing module pair is positioned adjacent to dummy object Object image data carry out illumination migration when, be configured as:

The 3D model of dummy object in virtual scene is tentatively rendered；

The photometric data for the object being positioned adjacent to dummy object and shadow data are migrated to preliminary based on intelligent algorithm To obtain destination virtual object on dummy object after rendering.

15. device according to claim 13 or 14, which is characterized in that the data processing module is also configured to

According to the attitude data of active user come use environment light rendering dummy object to obtain the preliminary wash with watercolours of user's current visual angle The image data of the dummy object of dye.

16. device as claimed in claim 14, which is characterized in that the secondary rendering is to the dummy object with illumination Image data carries out Shadow recognition to obtain the direction of illumination of dummy object, and according to the direction of illumination of dummy object and placement Plan-position renders the shadow image of dummy object.

17. a kind of computer readable storage medium, is stored with computer program, which is characterized in that the computer program is located When managing device operation, the processor perform claim requires method described in any one of 1-8.

18. a kind of computer equipment, the memory including processor and storage computer program, which is characterized in that the calculating When machine program is run by processor, the processor executes such as method of any of claims 1-8.

Technical field

The present invention relates to augmented reality fields, more particularly, are related to one kind and render virtual object in augmented reality The method and device of body.

Background technique

Augmented reality (Augmented Reality, abbreviation AR) is a kind of by real world information and virtual world information " seamless " integrated new technology, the technology is by the position of calculating video camera image and angle in real time and adds respective virtual mould Dummy model is loaded into real world and is interacted to realize by type.But to the wash with watercolours of dummy model in augmented reality Dye problem is always to influence the key of visual experience, and especially to the illumination render of dummy model, the influence to visual experience is most Greatly.

Summary of the invention

In view of the deficiencies of the prior art, the present invention provides a kind of in augmented reality renders the method and dress of dummy object It sets, is migrated the illumination of the Reference in reality scene on the dummy object into virtual scene by intelligent algorithm, So that dummy object is more really merged with reality scene.

According to an aspect of the present invention, a kind of method that dummy object is rendered in augmented reality, the method are provided The following steps are included: S1: obtaining the image data of reality scene；S2: according to the relevant information of the dummy object in virtual scene The image data of Reference is identified from the image data of the reality scene of acquisition；S3: will be joined using intelligent algorithm The photometric data for examining the image data of object is migrated into the image data of dummy object to obtain destination virtual object.

Preferably, the material parameters and/or dummy object that the relevant information of the dummy object includes dummy object are in void Location information in quasi- scene.

Preferably, the step S2 includes: the picture number of the reality scene according to the material parameters of dummy object from acquisition The image data of the generic object similar with dummy object material is identified in, or according to dummy object in virtual scene In location information the figure with the object of dummy object being positioned adjacent to is identified from the image data of the reality scene of acquisition As data.

Preferably, the step S3 includes: by the light of the image data of the generic object similar with dummy object material According in the image data of Data Migration to dummy object, or by the image data of the object being positioned adjacent to dummy object Photometric data is migrated into the image data of dummy object.

Preferably, the photometric data of the image data of the generic object similar with dummy object material is migrated to virtual Step in the image data of object includes: tentatively to be rendered to the 3D model of the dummy object in virtual scene；Based on people The photometric data of generic object is migrated and has illumination on the dummy object to after tentatively rendering to obtain by work intelligent algorithm The image data of dummy object；Secondary rendering is carried out to the image data of the dummy object with illumination to obtain target void Quasi- object.

Preferably, the photometric data of the image data for the object being positioned adjacent to dummy object is migrated to dummy object Step in image data includes: tentatively to be rendered to the 3D model of the dummy object in virtual scene；Based on artificial intelligence Algorithm migrates the photometric data for the object being positioned adjacent to dummy object and shadow data to the dummy object after preliminary rendering On to obtain destination virtual object.

Preferably, the step of dummy object in virtual scene is tentatively rendered includes: according to active user Attitude data come use environment light rendering dummy object to obtain the figure for the dummy object of user's current visual angle tentatively rendered As data.

Preferably, the image data to the dummy object with illumination carries out secondary rendering to obtain target void The step of quasi- object includes: to carry out Shadow recognition to the image data of the dummy object with illumination to obtain the light of dummy object According to direction, and the direction of illumination of the dummy object according to the acquisition and dummy object plan-position placed renders dummy object Shadow image.

According to another aspect of the present invention, a kind of device that dummy object is rendered in augmented reality, the dress are provided Setting includes: contextual data module, is configured as obtaining the image data of reality scene；Object identification module is configured as basis The relevant information of dummy object in virtual scene identifies the figure of Reference from the image data of the reality scene of acquisition As data；Data processing module is configured with intelligent algorithm and moves the photometric data of the image data of Reference It moves in the image data of dummy object to obtain destination virtual object.

Preferably, the material parameters and/or dummy object that the relevant information of the dummy object includes dummy object are in void Location information in quasi- scene.

Preferably, the object identification module is configured as: according to the material parameters of dummy object from the real field of acquisition The image data of the generic object similar with dummy object material is identified in the image data of scape, or according to dummy object Location information in virtual scene is identified from the image data of the reality scene of acquisition to be positioned adjacent to dummy object Object image data.

Preferably, the data processing module is configured as: by the figure of the generic object similar with dummy object material As the photometric data of data is migrated into the image data of dummy object, or by the object being positioned adjacent to dummy object The photometric data of image data is migrated into the image data of dummy object.

Preferably, described when carrying out illumination migration to the image data of the generic object similar with dummy object material Data processing module is configured as: tentatively being rendered to the 3D model of the dummy object in virtual scene；Based on artificial intelligence Algorithm migrates the photometric data of generic object on the dummy object to after tentatively rendering to obtain the virtual object for having illumination The image data of body；Secondary rendering is carried out to obtain destination virtual object to the image data of the dummy object with illumination Body.

Preferably, when carrying out illumination migration to the image data for the object being positioned adjacent to dummy object, at the data Reason module is configured as: tentatively being rendered to the 3D model of the dummy object in virtual scene；It will based on intelligent algorithm The photometric data and shadow data for the object being positioned adjacent to dummy object migrate on the dummy object to after tentatively rendering to obtain Destination virtual object out.

Preferably, the data processing module is also configured to according to the attitude data of active user come use environment light Dummy object is rendered to obtain the image data for the dummy object of user's current visual angle tentatively rendered.

Preferably, the secondary rendering is to carry out Shadow recognition to the image data of the dummy object with illumination to obtain The direction of illumination of dummy object, and render according to the plan-position of the direction of illumination of dummy object and placement the yin of dummy object Shadow image.

According to another aspect of the present invention, a kind of computer readable storage medium is provided, computer program is stored with, institute When stating computer program and being run by processor, the processor executes renders dummy object in augmented reality as described above Method.

According to another aspect of the present invention, a kind of computer equipment, including processor and storage computer program are provided Memory, when the computer program is run by processor, the processor executes the wash with watercolours in augmented reality as described above Contaminate the method for dummy object.

Detailed description of the invention

Carry out detailed description of the invention below in conjunction with attached drawing, features described above of the invention and other purposes, feature and Advantage will become apparent, in which:

Fig. 1 shows a kind of process of method that dummy object is rendered in augmented reality of embodiment according to the present invention Figure；

Fig. 2 shows a kind of block diagrams of device that dummy object is rendered in augmented reality of embodiment according to the present invention.

In the accompanying drawings, identical label will be understood to refer to identical element, feature and structure.

Specific embodiment

Hereinafter, the embodiment of the present invention is described in detail with reference to the attached drawings.Wherein, identical label always shows identical portion Part.

Fig. 1 is the process for showing a kind of method that dummy object is rendered in augmented reality of embodiment according to the present invention Figure.

As shown in Figure 1, obtaining the image data of reality scene in step S1.According to an embodiment of the invention, specifically, The image data of reality scene is obtained by capture apparatus such as camera, video camera etc. and deposits the image data that shooting obtains Storage, wherein the image data of acquisition includes image data and/or video data.

Next, in step S2, according to the relevant information of the dummy object in virtual scene from the reality scene of acquisition The image data of Reference is identified in image data.Specifically, the relevant information of dummy object includes the material of dummy object The location information of matter parameter and/or dummy object in virtual scene therefore can be according to the material parameters of dummy object from acquisition Reality scene image data in identify the image data of the generic object similar with dummy object material, or according to Location information of the dummy object in virtual scene identifies and dummy object from the image data of the reality scene of acquisition The image data for the object being positioned adjacent to.Wherein, the material parameters of dummy object include highlight strength, bloom area, self-luminous The distinctive illumination parameter of the objects such as coefficient, reflectivity, refractive index, glossiness, roughness, material is similar to be referred to and dummy object The case where material parameters are identical or difference very little, location information of the dummy object in virtual scene are the three-dimensional of dummy object Coordinate information.It should be understood that the citing of the above-mentioned material parameters for dummy object is only illustrative examples, the present invention is adoptable The type of material parameters is without being limited thereto.

According to an embodiment of the invention, assume that the image data of reality scene obtained in step S1 is picture T, then basis The material parameters or location information of dummy object in virtual scene identify the image data of the Reference in picture T. When identifying the image data of Reference according to the material parameters of the dummy object in virtual scene, if the material of dummy object Matter parameter is the high optical parameter of PBR material system, then identifies that material parameters in picture T are all the high optical parameter of PBR material system Or the generic object of the high optical parameter of the Vray material system similar with the high optical parameter of PBR material (uses reference substance below Body A is indicated), and using the image data of the generic object as the image data of reference substance object.Here, for generic Mode or the intelligent algorithm of feature point extraction can be used to be identified in the identification of object.When according in virtual scene When image data of the location information of dummy object to identify Reference, it can only identify that dummy object institute placement location is attached Close object (being indicated below with Reference B), and to the carry out material of the Reference of identification after identifying Reference The identification of matter parameter.Here, it object refers to neighbouring with dummy object and is put down same near dummy object institute placement location Object in the image data of reality scene on face.It should be understood that the above-mentioned citing for recognition methods is only illustrative examples, The adoptable recognition methods of the present invention is without being limited thereto.

In step S3, illumination migration is carried out to obtain target void using image data of the intelligent algorithm to Reference Quasi- object.Specifically, can be used the image data of the intelligent algorithm pair generic object similar with dummy object material into Row illumination migration carries out illumination migration to the image data of the object being positioned adjacent to dummy object to obtain target void Quasi- object.Here, common intelligent algorithm may include artificial neural network (Artificial Neural Network) class Algorithm, Bayes's (Bayesin) class algorithm, decision tree (Decision Tree) class algorithm, linear classifier (Linear Classifier) class algorithm etc..When the image using the intelligent algorithm pair generic object similar with dummy object material When data carry out illumination migration, first the 3D model of the dummy object in virtual scene is tentatively rendered, then, based on artificial The photometric data of generic object is migrated and has illumination on the dummy object to after tentatively rendering to obtain by intelligent algorithm model Dummy object image data, then to illumination dummy object image data carry out secondary rendering with obtain target void Quasi- object.When the image data progress illumination migration using intelligent algorithm pair and the object of dummy object being positioned adjacent to When, after the 3D model to the dummy object in virtual scene tentatively render, intelligent algorithm model can be directly based upon By the photometric data for the object being positioned adjacent to dummy object and shadow data migrate on the dummy object after preliminary rendering with Obtain destination virtual object.Here, preliminary rendering refers to virtual come the rendering of use environment light according to the attitude data of active user Object is to obtain the image data of the dummy object of user's current visual angle, wherein the attitude data of user includes user and virtual The three-dimensional coordinate of positional relationship, user between object itself, user the action datas such as gesture.

According to an embodiment of the invention, when the image data to the generic object similar with dummy object material carries out light When according to migration, such as the example above can make the image data of the dummy object after the image data of Reference A and preliminary rendering For the input of convolutional neural networks model trained in advance, the then output (i.e. the result of illumination migration) of convolutional neural networks model Secondary rendering then is carried out to obtain mesh to the dummy object with illumination for the image data of the dummy object with illumination Mark dummy object.Here, convolutional neural networks can realize that the illumination by the object G for having illumination is migrated to the object F of no light And export the process of object F, wherein have the object G of illumination can be similar with the material of the object F of no light.Training in advance Convolutional neural networks model is obtained by carrying out a large amount of repetitive exercise to above-mentioned convolutional neural networks, convolutional neural networks The output of model is the image data of the object F with illumination.Secondary rendering refers to the figure to the output of convolutional neural networks model Dummy object as having illumination in data carries out Shadow recognition to obtain the direction of illumination and shadow data of dummy object, and according to The shadow image of dummy object is rendered according to the direction of illumination of dummy object, shadow data and the plan-position of placement.According to this The embodiment of invention, when the image data to the object being positioned adjacent to dummy object carries out illumination migration, such as above-mentioned act Example, by the image data of the dummy object after the image data of Reference B and preliminary rendering and the material ginseng of two objects The input as another convolutional neural networks model of training in advance is counted, then output (the i.e. light of the convolutional neural networks model According to the result of migration) be destination virtual object image data.Using the depth convolutional neural networks model to dummy object into Row illumination migration is the virtual object moved to the photometric data of Reference B and the shadow data of identification together after preliminary rendering On body, it therefore, there is no need to carry out the image data that secondary rendering has just obtained destination virtual object.Here, convolutional neural networks Can be achieved together to migrate the shadow data of the object G of illumination and the identification of the object G for having illumination on the object F of no light simultaneously Export the process of object F, wherein have the object G of illumination can be different from the material of the object F of no light.The convolutional Neural net Network model is obtained by carrying out a large amount of repetitive exercise to above-mentioned convolutional neural networks, and the convolutional neural networks model Output is the image data of the object F with illumination and shade, the i.e. image data of destination virtual object.

Fig. 2 is the frame for showing a kind of device that dummy object is rendered in augmented reality of embodiment according to the present invention Figure.

As shown in Fig. 2, the device 200 for rendering dummy object in augmented reality includes contextual data module 201, object Body identification module 202 and data processing module 203.Specifically, contextual data module 201 is configured as obtaining the figure of reality scene As data, object identification module 202 is configured as the reality according to the relevant information of the dummy object in virtual scene from acquisition The image data of Reference is identified in the image data of scene, data processing module 203 is configured with artificial intelligence Algorithm carries out illumination to the image data of Reference and migrates to obtain destination virtual object.

Contextual data module 201 be configured to include capture apparatus and storage equipment, wherein capture apparatus such as camera or Video camera etc. is used to shoot the image data of reality scene, and storage equipment is used to store the image data that shooting obtains, Wherein, the image data of shooting includes image data and/or video data.

Object identification module 202 is according to the relevant information of the dummy object in virtual scene from contextual data module 201 The image data of Reference is identified in the image data of the reality scene of acquisition, wherein the relevant information packet of dummy object Include the location information of the material parameters and/or dummy object of dummy object in virtual scene.Specifically, object identification module 202 can identify from the image data of the reality scene obtained in contextual data module 201 according to the material parameters of dummy object The generic object similar with dummy object material, can also be according to dummy object in virtual scene as Reference out Location information identifies and the position of dummy object neighbour from the image data of the reality scene obtained in contextual data module 201 Close object is as Reference.

Data processing module 203 obtains Reference to identification in object identification module 202 based on intelligent algorithm Image data carries out illumination migration to obtain destination virtual object.Specifically, data processing module 203 to dummy object material When the image data of similar generic object carries out illumination migration, first the 3D model of the dummy object in virtual scene is carried out Preliminary rendering, and migrated the photometric data of generic object to the virtual object after preliminary rendering based on intelligent algorithm model On body, then, then destination virtual object can just be shown to the dummy object progress secondary rendering after illumination migration.Wherein, tentatively Rendering refers to according to the attitude data of active user come use environment light rendering dummy object to obtain the void of user's current visual angle The image data of quasi- object, the attitude data of user include positional relationship between user and dummy object, user itself three Tie up the data such as coordinate and the gesture of user.Secondary rendering, which refers to, carries out Shadow recognition to the dummy object with illumination to obtain void The direction of illumination and shadow data of quasi- object, and the plan-position of the direction of illumination according to dummy object, shadow data and placement To render the shadow image of dummy object.In addition, data processing module 203 is to the object being positioned adjacent to dummy object When image data carries out illumination migration, equally first the 3D model of the dummy object in virtual scene tentatively can be rendered, so Afterwards, the photometric data for the object being positioned adjacent to dummy object and shadow data are migrated to first based on intelligent algorithm model To obtain destination virtual object on dummy object after step rendering.

The method and device that dummy object is rendered in augmented reality of embodiment according to the present invention can be by real field The illumination of Reference in scape migrates on the dummy object into virtual scene, melts dummy object really with reality scene Unify, meets requirement of the user to visual experience.

Although the present invention, art technology has been shown and described referring to certain exemplary embodiments of the invention Personnel will be understood that, can be into the case where not departing from the spirit and scope of the present invention being defined by the claims and their equivalents Various changes on row various forms and details.