阅读说明：本技术 一种用于剧本杀环境的投影系统及方法 (Projection system and method for script killing environment ) 是由 叶晓青 冯兵 朱登明 朱正刚 于 2021-09-07 设计创作，主要内容包括：本发明实施例公开了一种用于剧本杀环境的沉浸式投影系统及方法。该剧本杀环境的沉浸式投影系统及方法采用三维场景的互动投影并将其应用至剧本杀环境中,从而提高真实的视觉体验、配合互动以使用户得到身临其境的沉浸式体验,且能够降低场景布置的成本和/或浪费。(The embodiment of the invention discloses an immersive projection system and method for a script killing environment. The immersive projection system and the immersive projection method for the script killing environment adopt interactive projection of the three-dimensional scene and apply the interactive projection to the script killing environment, so that real visual experience is improved, interaction is matched to enable a user to obtain immersive experience, and cost and/or waste of scene arrangement can be reduced.)

1. An immersive projection method for a screenplay killing environment, comprising:

step S1: the projection module projects the structured light image information into a real environment;

step S2: the image acquisition module acquires the real environment image projected with the structured light image information through the step S1 and transmits the image to the processing module;

step S3: the processing module acquires a plurality of real environment images projected with structured light image information, reconstructs three-dimensional model information by adopting a three-dimensional reconstruction algorithm, and transmits the three-dimensional model information to the rendering module;

step S4: the character action perception module collects three-dimensional point cloud data and image data of characters in real time and transmits the three-dimensional point cloud data and the image data to the processing module;

step S5: the processing module judges the action type of the person or/and the position information of the person based on the three-dimensional point cloud data and the image data, converts the action type of the person or/and the position information of the person into texture data, and transmits the texture data to the rendering module;

step S6: the rendering module projects the texture data to the three-dimensional model by adopting a UV mapping mode according to the three-dimensional model information and the texture data; then, a final image is rendered through a virtual camera in a rendering module, and the final image is transmitted to a projection module;

step S7: and the projection module projects the final image into a real environment in a projection mode.

2. The immersive projection method for a screenplay killing environment of claim 1, wherein in step S1, the structured light image is transmitted by a processing module to the projection module via an image interface.

3. The immersive projection method for the screenplay killing environment of claim 1, wherein the processing module calculates the projection parameters by calibration and transmits the projection parameters to the projection module in step S6.

4. The immersive projection method for the screenplay killing environment of claim 3, wherein the step S7 is implemented by combining the projection parameters with a projection module to project the final image into the real environment in a projection manner.

5. The immersive projection method for a screenplay killing environment of claim 1, wherein in step S2, the processing module completes parameter configuration of the image acquisition module and controls image acquisition timing of the image acquisition module.

6. An immersive projection system for a screenplay killing environment, the immersive projection system for the screenplay killing environment comprising:

the projection module is used for projecting the structured light image information into a real environment;

the image acquisition module is connected with the projection module and used for acquiring an output image of the projection module and transmitting the image to the processing module;

the figure action perception module is connected with the processing module and used for acquiring three-dimensional point cloud data and image data of a figure in real time and transmitting the three-dimensional point cloud data and the image data to the processing module;

the processing module is used for constructing three-dimensional model information according to the image acquired by the image acquisition module, judging the action type of the person or/and the position information of the person based on the three-dimensional point cloud data and the image data, and converting the action type of the person or/and the position information of the person into texture data available for the rendering module;

the rendering module is connected with the processing module and used for projecting the texture data onto the three-dimensional model by adopting a UV mapping mode according to the three-dimensional model information and the texture data output by the processing module; then, rendering a final image through a virtual camera in the rendering module; the rendering module is connected with the projection module, and the projection module projects the final image into a real environment in a projection mode.

7. The immersive projection system of claim 6, wherein the image capture module is connected to the processing module via a communication interface, the human motion perception module is connected to the processing module via a communication interface, the processing module is interactively connected to the rendering module via a memory, and the rendering module is connected to the projection module via an image interface.

8. The immersive projection system for a screenplay killing environment of claim 6, wherein the human motion perception module is any one of a three-dimensional camera, a radar, or a TOF device.

9. The immersive projection system for a screenplay killing environment of claim 6, wherein the projection module comprises a plurality of camera devices, the plurality of camera devices being respectively disposed at a central location and at four east, south, west and north orientations in the real environment.

10. The immersive projection system for a screenplay killing environment of claim 9, wherein the projection module comprises a projection rack assembly positioned at a top of the real environment, the plurality of camera devices being disposed on the projection rack assembly.

Technical Field

The invention relates to the technical field of immersive entertainment experience, in particular to a projection system and a projection method for a script killing environment.

Background

Since 2016, "script killer" became a popular table-top role-playing game today in China. "Theater" is characterized by the immersive and realistic sensation of the experience. At present, a venue providing a 'script killing' service mostly takes a real object scene as a main part, a real scene matching the script needs to be created, and the input cost is very high; meanwhile, as the novelty of the script determines the experience of the script killing, all scenes need to be rearranged frequently according to the change of the script, and great waste is caused. In addition, the real scene is generally only used as a background, and if interaction with the scene is needed, interaction props need to be added in the scene, so that investment is further increased.

Therefore, in view of the above technical problems, there is a need for an immersive projection system and method for a screenplay killing environment, so that the cost of scene arrangement can be reduced without affecting the immersive and realistic effects of the screenplay killing experience.

Disclosure of Invention

It is therefore an object of the present invention to provide an immersive projection system and method for a screenplay environment. The immersive projection system or method for the script killing environment provided by the embodiment of the invention adopts interactive projection of a three-dimensional scene and applies the interactive projection to the script killing environment, so that the real visual experience is improved, the user obtains immersive experience by matching the interactive projection, and the cost and/or waste of scene arrangement can be reduced.

In order to achieve the above purpose, the technical solutions provided by the embodiments of the present invention are as follows: an immersive projection method for a screenplay killing environment includes step S1: the projection module projects the structured light image information into a real environment; step S2: the image acquisition module acquires the real environment image projected with the structured light image information through the step S1 and transmits the image to the processing module; step S3: the processing module acquires a plurality of real environment images projected with structured light image information, reconstructs three-dimensional model information by adopting a three-dimensional reconstruction algorithm, and transmits the three-dimensional model information to the rendering module; step S4: the character action perception module collects three-dimensional point cloud data and image data of characters in real time and transmits the three-dimensional point cloud data and the image data to the processing module; step S5: the processing module judges the action type of the person or/and the position information of the person based on the three-dimensional point cloud data and the image data, converts the action type of the person or/and the position information of the person into texture data, and transmits the texture data to the rendering module; step S6: the rendering module projects the texture data to the three-dimensional model by adopting a UV mapping mode according to the three-dimensional model information and the texture data; then, a final image is rendered through a virtual camera in a rendering module, and the final image is transmitted to a projection module; step S7: and the projection module projects the final image into a real environment in a projection mode.

As a further improvement of the invention, the structured light image is transmitted by the processing module to the projection module via an image interface.

As a further improvement of the present invention, step S6 further includes that the processing module calculates the projection parameters by using a calibration method, and transmits the projection parameters to the projection module.

As a further improvement of the present invention, in step S7, the projection module projects the final image into a real environment by using a projection method in combination with the projection parameters.

As a further improvement of the present invention, in step S2, the processing module completes parameter configuration of the image capturing module and controls image capturing timing of the image capturing module.

The embodiment of the invention also provides an immersive projection system for the script killing environment. Wherein, this immersive projection system for script killing environment includes: the projection module is used for projecting the structured light image information into a real environment; the image acquisition module is connected with the projection module and used for acquiring an output image of the projection module and transmitting the image to the processing module; the figure action perception module is connected with the processing module and used for acquiring three-dimensional point cloud data and image data of a figure in real time and transmitting the three-dimensional point cloud data and the image data to the processing module; the processing module is used for constructing three-dimensional model information according to the image acquired by the image acquisition module, judging the action type of the person or/and the position information of the person based on the three-dimensional point cloud data and the image data, and converting the action type of the person or/and the position information of the person into texture data available for the rendering module; the rendering module is connected with the processing module and used for projecting the texture data onto the three-dimensional model by adopting a UV mapping mode according to the three-dimensional model information and the texture data output by the processing module; then, rendering a final image through a virtual camera in the rendering module; the rendering module is connected with the projection module, and the projection module projects the final image into a real environment in a projection mode.

As a further improvement of the present invention, the image acquisition module is connected to the processing module through a communication interface, the character motion perception module is connected to the processing module through a communication interface, the processing module is interactively connected to the rendering module through a memory, and the rendering module is connected to the projection module through an image interface.

As a further improvement of the invention, the human motion perception module is any one of a three-dimensional camera, a radar or a TOF device.

As a further improvement of the present invention, the projection module includes a plurality of photographing devices respectively disposed at a middle position and four orientations of east, south, west, and north in the real environment.

As a further improvement of the invention, the projection module comprises a projection bracket assembly positioned at the top of the real environment, and the plurality of photographic devices are arranged on the projection bracket assembly

The invention has the following advantages:

the immersive projection method and the immersive projection system for the script killing environment provided by the embodiment of the invention adopt the projection module to project the structured light image to the real scene, the image acquisition module acquires the real scene projected with the structured light image and transmits the real scene to the processing module, the processing module reconstructs a three-dimensional model of the real scene through the image and transmits the three-dimensional model to the rendering module, the rendering module maps texture data to be displayed to the three-dimensional model in a uv chartlet mode, a rendering result is transmitted to the projection module in a graphic display card mode, and the projection module projects the result to the real scene. Further, the character action perception module collects point cloud data and depth map data of characters at the same time, the point cloud data and the depth map data are transmitted to the processing module, the processing module judges and identifies action types of the characters, different textures are switched and mapped to the three-dimensional model according to different action types, and the three-dimensional model is finally displayed in a real scene.

The immersive projection method and the immersive projection system for the script killing environment provided by the embodiment of the invention can complete three-dimensional projection aiming at a real scene and build a dazzling simulation scene, and meanwhile, people can interact with the scene to realize immersive script killing experience; compared with the prior art, the method has the advantages that the scene is really built, and the cost is effectively reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of an immersive projection method for a screenplay killing environment according to an embodiment of the present invention;

fig. 2 is a block diagram of an immersive projection system for a screenplay environment according to an embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, a schematic flowchart of an immersive projection method for a screenplay environment according to an embodiment of the present invention is provided. In this embodiment, the flow diagram of the immersive projection method for the screenplay killing environment includes seven steps, and the specific content of each step is as follows.

Step S1: the projection module projects the structured light image information into a real environment. In this embodiment, the structured light image is stored in a processing module, which performs the configuration of the parameters for the structured pattern and controls the projection module to project the structured light pattern; the structured light image is transmitted to the projection module by the processing module through an image interface.

Step S2: the image capturing module captures the real environment image projected with the structured light image information through step S1 and transmits the image to the processing module. The image acquisition module comprises an image sensor, and the processor module is used for completing the configuration of the image sensor and controlling the time of image acquisition. In this embodiment, the image sensor may specifically be a camera or any other device capable of acquiring RGB image information. According to the projection module and the image acquisition module, the relative position relation between the projection module and the image acquisition equipment is automatically calculated through the processing module by acquiring the data of the image, and the relative position relation is used for finally matching the projection picture with the real scene.

Step S3: the processing module obtains a plurality of real environment images projected with structured light image information, reconstructs three-dimensional model information by adopting a three-dimensional reconstruction algorithm, and transmits the three-dimensional model information to the rendering module.

Step S4: the character action perception module collects three-dimensional point cloud data and image data of characters in real time and transmits the three-dimensional point cloud data and the image data to the processing module. In this embodiment, the human motion perception module is used for acquiring a point cloud and an RGB image device of a human. Specifically, the human motion perception module may be any one of a 3D camera, a radar, or a TOF device.

In this embodiment, the step S4 is not logically related to the steps S1 to S3, and the sequence of executing the steps is not limited herein, and may be executed synchronously or sequentially without any order.

Step S5: the processing module judges the action type of the person or/and the position information of the person based on the three-dimensional point cloud data and the image data, converts the action type of the person or/and the position information of the person into texture data, and transmits the texture data to the rendering module.

In this embodiment, the processing module includes at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any of the immersive projection methods for a screenplay killing environment.

Step S6: the rendering module projects the texture data to the three-dimensional model by adopting a UV mapping mode according to the three-dimensional model information and the texture data; and then, rendering a final image through a virtual camera in the rendering module, and transmitting the final image to the projection module. The rendering module comprises a rendering engine running on the processing module.

In order to prevent the situation that the projection picture of the projection module and the surface of the three-dimensional object of the real scene are not matched in step S7, preferably, in this step, the processing module calculates the projection parameters by calibration and transmits the projection parameters to the projection module.

Step S7: and the projection module projects the final image into a real environment in a projection mode. Preferably, the projection module projects the final image into a real environment in a projection manner in combination with the projection parameters.

The immersive projection method for the script killing environment provided by the embodiment of the invention projects the structured light image to the real scene through the projection module, the image acquisition module acquires the real scene projected with the structured light image and transmits the real scene to the processing module, the processing module reconstructs a three-dimensional model of the real scene through the image and transmits the three-dimensional model to the rendering module, the rendering module maps texture data to be displayed to the three-dimensional model in a uv mapping mode, a rendering result is transmitted to the projection module in a graphic display card mode, and the projection module projects the result to the real scene. Further, the character action perception module collects point cloud data and depth map data of characters at the same time, the point cloud data and the depth map data are transmitted to the processing module, the processing module judges and identifies action types of the characters, different textures are switched and mapped to the three-dimensional model according to different action types, and the three-dimensional model is finally displayed in a real scene.

As shown in fig. 2, an embodiment of the present invention provides a module schematic diagram of an immersive projection system for a screenplay environment.

In this embodiment, an immersive projection system 100 for a screenplay killing environment includes: a projection module 10 for projecting structured light image information into a real environment; the image acquisition module 20 is connected with the projection module 10 and used for acquiring an output image of the projection module 10 and transmitting the image to the processing module 30; the figure action perception module 40 is connected with the processing module 30, and the figure action perception module 40 is used for acquiring three-dimensional point cloud data and image data of a figure in real time and transmitting the three-dimensional point cloud data and the image data to the processing module 30; the processing module 30 is configured to construct three-dimensional model information according to the image obtained by the image acquisition module 20, and is configured to determine an action type of the person or/and position information of the person based on the three-dimensional point cloud data and the image data, and convert the action type of the person or/and the position information of the person into texture data usable by the rendering module 50; the rendering module 50 is connected to the processing module 30, and the rendering module 50 is configured to project the texture data onto the three-dimensional model by using a UV mapping manner according to the three-dimensional model information and the texture data output by the processing module 30; then, rendering a final image through a virtual camera in the rendering module; the rendering module 50 is connected to the projection module 10, and the projection module 10 projects the final image into a real environment in a projection manner.

In this embodiment, the image capturing module 20 is connected to the processing module 30 through a communication interface, the human motion sensing module 40 is connected to the processing module 30 through a communication interface, the processing module 30 is interactively connected to the rendering module 50 through a memory, and the rendering module 50 is connected to the projection module 10 through an image interface.

In this embodiment, the human motion perception module 40 is any one of a three-dimensional camera, a radar, or a TOF device. The character action perception module 40 perceives the position information of the character, controls different textures to be mapped to the three-dimensional model according to different positions, and finally displays the three-dimensional model in a real scene.

In this embodiment, the projection module 10 includes a plurality of photographing devices which are respectively disposed at a middle position and four orientations of east, south, west, and north in the real environment. The plurality of photographing apparatuses includes at least 5 photographing apparatuses. Preferably, the projection module 10 includes a projection bracket assembly positioned at the top in the real environment, on which the plurality of camera devices are disposed.

The immersive projection method and the immersive projection system for the script killing environment provided by the embodiment of the invention can complete three-dimensional projection aiming at a real scene and build a dazzling simulation scene, and meanwhile, people can interact with the scene to realize immersive script killing experience; compared with the prior art, the method has the advantages that the scene is really built, and the cost is effectively reduced.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.

Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.