阅读说明：本技术 一种全息展示方法及装置 (Holographic display method and device ) 是由 谢国勇 刘嘉兴 许秋子 于 2021-10-09 设计创作，主要内容包括：本申请涉及一种全息展示方法及装置,方法包括：获取第一用户的视点信息,其中,视点信息示出第一用户双目的位置以及第一用户双目的视角,第一用户是持有交互控制笔的用户；获取交互控制笔的位姿信息；确定视角和指向之间的第一夹角；若第一夹角大于第一角度阈值,则将头戴式显示装置的输出频率调整至第一频率,若第一夹角不大于第一角度阈值,则将头戴式显示装置的输出频率调整至第二频率,其中,第一频率大于第二频率。在本说明书中,根据第一用户的视角和交互控制笔的指向之间的第一夹角,调整头戴式显示装置的输出频率,有利于第一用户辨识出其在全息影像中希望进行交互的部分内容。本申请中的方法适用于针对多人的教学场景。(The application relates to a holographic display method and a device, wherein the method comprises the following steps: acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose visual angle of the first user, and the first user is a user holding an interactive control pen; acquiring pose information of the interactive control pen; determining a first included angle between a visual angle and a pointing direction; if the first included angle is larger than the first angle threshold, the output frequency of the head-mounted display device is adjusted to a first frequency, and if the first included angle is not larger than the first angle threshold, the output frequency of the head-mounted display device is adjusted to a second frequency, wherein the first frequency is larger than the second frequency. In this specification, the output frequency of the head-mounted display device is adjusted according to a first included angle between the viewing angle of the first user and the pointing direction of the interactive control pen, so that the first user can recognize a part of the content that the first user wants to interact with in the hologram. The method is suitable for teaching scenes for multiple persons.)

1. A holographic display method, said method being based on a holographic display system, said holographic display system comprising: the display device comprises display processing equipment, a sand table type display device, a head-mounted display device and an interactive control pen; the display processing equipment is electrically connected with the sand table type display device, the head-mounted display device and the interactive control pen respectively; the holographic display method is executed by the display processing equipment, and comprises the following steps:

acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the first user, and the first user is a user holding the interactive control pen;

acquiring pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;

determining a first angle between the viewing angle and the pointing direction;

if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

2. The method of claim 1, wherein adjusting the output frequency of the head-mounted display device to a first frequency if the first angle is greater than a first angle threshold, wherein adjusting the output frequency of the head-mounted display device to a second frequency if the first angle is less than the first angle threshold, and wherein before the first frequency is greater than the second frequency, the method further comprises:

acquiring the number of users, wherein the users comprise the first user and a second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen;

if the number of the users is larger than a number threshold, adjusting the value of the first frequency to a first value, and if the number of the users is not larger than the number threshold, adjusting the value of the first frequency to a second value, wherein the first value is smaller than the second value.

3. The method of claim 1, wherein adjusting the output frequency of the head-mounted display device to a first frequency if the first angle is greater than a first angle threshold, wherein adjusting the output frequency of the head-mounted display device to a second frequency if the first angle is less than the first angle threshold, and wherein before the first frequency is greater than the second frequency, the method further comprises:

acquiring viewpoint information of each second user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen;

determining a density center of distribution of the second user dual-purpose positions in a specified environment, wherein the specified environment is the environment where the sand table type display device is located;

determining a connecting line between the density center and a designated point as a designated line, wherein the designated point is an intersection point of the pointing direction of the interactive control pen and the holographic image;

if a second included angle between the direction and the specified line is larger than a second angle threshold, adjusting the output frequency of the sand table type display device to a third frequency, and if the second included angle is not larger than the second angle threshold, adjusting the output frequency of the sand table type display device to a fourth frequency, wherein the third frequency is larger than the fourth frequency.

4. The method of claim 1, wherein the viewpoint information comprises six degrees of freedom information of a head mounted display device worn by the first user, wherein the viewpoint information is derived from a viewpoint information tracking algorithm.

5. The method according to claim 1, wherein the pose information of the interactive control pen comprises six-degree-of-freedom information of the interactive control pen, wherein position 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to image information acquired by a mobile auxiliary camera, and rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to data acquired by an IMU (inertial measurement unit) module in the interactive control pen, and the mobile auxiliary camera is arranged on the sand-disc type display device.

6. The method of claim 1, wherein the head-mounted display device comprises shutter-type active three-dimensional glasses and at least three reflective mark points embedded on a housing structure of the shutter-type active three-dimensional glasses, and the shutter-type active three-dimensional glasses are used for receiving the hologram transmitted by the sand-tray type display device, so as to obtain correct left and right eye pictures.

7. The method according to claim 1, wherein a first pressure sensor is disposed on the head-mounted display device, and when the head-mounted display device is worn on the head of a user, the first pressure sensor generates a first signal and sends the first signal to the display processing device; the interactive control pen is internally provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing equipment; the acquiring of the viewpoint information of the first user includes:

and if the first signal is detected at a specified time and the second signal is detected within a specified time range from the specified time, acquiring viewpoint information of the first user.

8. A holographic display, characterized in that the device comprises:

a first acquisition module configured to: acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the first user, and the first user is a user holding the interactive control pen;

a second acquisition module configured to: acquiring pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;

a first angle determination module configured to: determining a first angle between the viewing angle and the pointing direction;

a first adjustment module configured to: if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the holographic display method of any of claims 1 to 7 when executing a program stored in the memory.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the holographic display method of any of claims 1 to 7.

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a holographic display method and apparatus.

Background

A Holographic Display technology (Front-Projected Holographic Display), also called a virtual imaging technology, is a technology for recording and reproducing a real three-dimensional image of an object by using interference and diffraction principles, and has the advantages of satisfying the entire perception of human vision, and enabling a viewer to view the three-dimensional image without using an auxiliary device such as a helmet or glasses. With the continuous development of display technology, holographic display technology has gained more and more attention.

In the application scenes such as cinema, science and technology museum, Virtual Reality (VR) helmet and the like, the arc-shaped or annular holographic image can enable the viewer to obtain the panoramic effect, so that the viewing experience of the viewer is greatly improved, and the viewer has the feeling of being personally on the scene. However, in the prior art, there are technical solutions that enable a user to interact with a hologram.

Disclosure of Invention

The application provides a holographic display method and a holographic display device, which are used for solving the problem that in the prior art, interaction between a user and a holographic image is difficult to realize.

In a first aspect, the present application provides a holographic display method, the method being based on a holographic display system, the holographic display system comprising: the display device comprises display processing equipment, a sand table type display device, a head-mounted display device and an interactive control pen; the display processing equipment is electrically connected with the sand table type display device, the head-mounted display device and the interactive control pen respectively; the holographic display method is executed by the display processing equipment, and comprises the following steps:

acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the first user, and the first user is a user holding the interactive control pen;

acquiring pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;

determining a first angle between the viewing angle and the pointing direction;

if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

In an optional embodiment of the present specification, if the first included angle is greater than a first angle threshold, the output frequency of the head-mounted display device is adjusted to a first frequency, and if the first included angle is smaller than the first angle threshold, the output frequency of the head-mounted display device is adjusted to a second frequency, before the first frequency is greater than the second frequency, the method further includes:

acquiring the number of users, wherein the users comprise the first user and a second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen;

if the number of the users is larger than a number threshold, adjusting the value of the first frequency to a first value, and if the number of the users is not larger than the number threshold, adjusting the value of the first frequency to a second value, wherein the first value is smaller than the second value.

In an optional embodiment of the present specification, if the first included angle is greater than a first angle threshold, the output frequency of the head-mounted display device is adjusted to a first frequency, and if the first included angle is smaller than the first angle threshold, the output frequency of the head-mounted display device is adjusted to a second frequency, before the first frequency is greater than the second frequency, the method further includes:

acquiring viewpoint information of each second user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen;

determining a density center of distribution of the second user dual-purpose positions in a specified environment, wherein the specified environment is the environment where the sand table type display device is located;

determining a connecting line between the density center and a designated point as a designated line, wherein the designated point is an intersection point of the pointing direction of the interactive control pen and the holographic image;

if a second included angle between the direction and the specified line is larger than a second angle threshold, adjusting the output frequency of the sand table type display device to a third frequency, and if the second included angle is not larger than the second angle threshold, adjusting the output frequency of the sand table type display device to a fourth frequency, wherein the third frequency is larger than the fourth frequency.

In an alternative embodiment of the present description, the viewpoint information comprises six-degree-of-freedom information of a head mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm.

In an optional embodiment of the present specification, the pose information of the interactive control pen includes six-degree-of-freedom information of the interactive control pen, where position 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to image information acquired by a mobile auxiliary camera, rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to data acquired by an IMU module in the interactive control pen, and the mobile auxiliary camera is disposed on the sand table display device.

In an alternative embodiment of the present disclosure, the head-mounted display device includes shutter-type active three-dimensional glasses and at least three reflective mark points embedded on a housing structure of the shutter-type active three-dimensional glasses, and the shutter-type active three-dimensional glasses are used for receiving the holographic images transmitted by the sand-tray-type display device, so as to obtain correct left and right eye pictures.

In an optional embodiment of the present specification, a first pressure sensor is disposed on the head-mounted display device, and when the head-mounted display device is worn on the head of a user, the first pressure sensor generates a first signal and sends the first signal to the display processing apparatus; the interactive control pen is internally provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing equipment; the acquiring of the viewpoint information of the first user includes:

and if the first signal is detected at a specified time and the second signal is detected within a specified time range from the specified time, acquiring viewpoint information of the first user.

In a second aspect, the present application provides a holographic display comprising:

a first acquisition module configured to: acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the first user, and the first user is a user holding the interactive control pen;

a second acquisition module configured to: acquiring pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;

a first angle determination module configured to: determining a first angle between the viewing angle and the pointing direction;

a first adjustment module configured to: if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the steps of any one of the holographic display methods in the first aspect when executing the program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the holographic display methods of the first aspect.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:

in the description, the output frequency of the head-mounted display device is adjusted according to a first included angle between the visual angle of the first user and the pointing direction of the interactive control pen, and when the first included angle is larger than a first angle threshold, it indicates that a large position difference and/or angle difference exists between partial contents which the first user wants to interact in the holographic image, which may cause the first user to be unable to accurately select the partial contents which the first user wants to interact in the holographic image. Under the condition that the first included angle is not larger than the first angle threshold, the condition that the first user and the part of content which the first user wants to interact have good interaction is indicated, at the moment, the output frequency of the head-mounted display device is reduced, so that more users can participate in the observation of the holographic image, and the reasonable distribution of holographic image display resources is facilitated. The method is suitable for teaching scenes for multiple persons.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic view of a scene involved in a holographic display process according to an embodiment of the present disclosure;

fig. 2 is a schematic flow chart of a holographic display process according to an embodiment of the present disclosure;

FIG. 3 is a schematic view of a holographic display device corresponding to a part of the process of FIG. 2;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The process in this specification is based on a holographic display system comprising: the display device comprises display processing equipment, a sand table type display device, a head-mounted display device and an interactive control pen; the display processing device is electrically connected with the sand table type display device, the head-mounted display device and the interactive control pen respectively, and an exemplary scene is as shown in fig. 1. The head-mounted display device can correspond to a user one by one. Alternatively, different head mounted display devices output pictures to the user with the same frequency. In the case where multiple users (users not two or more) view holograms displayed on the sand-tray type display device at the same time, the display processing apparatus outputs a display signal to each head-mounted display device separately.

The method in the specification is suitable for teaching scenes, and under the condition that the users are multiple users, one of the multiple users is a teacher, and the other users are students.

In a scenario where multiple users watch a hologram displayed on a sand-tray type display device at the same time, the user wearing the head-mounted display device and holding the interactive control pen is a first user, and the user wearing the head-mounted display device and not holding the interactive control pen is a second user.

As shown in fig. 2, the holographic display method and apparatus in this specification includes the following steps:

s200: viewpoint information of a first user is acquired.

In this specification, the viewpoint information shows the first user dual-purpose position and the first user dual-purpose angle of view, the first user being a user holding the interactive control pen.

In an alternative embodiment of the present description, the viewpoint information comprises six-degree-of-freedom information of a head mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm. Optionally, the head-mounted display device includes shutter-type active three-dimensional glasses and at least three reflective mark points embedded on a housing structure of the shutter-type active three-dimensional glasses, and the shutter-type active three-dimensional glasses are used for receiving the hologram transmitted by the sand-tray-type display device, so as to obtain correct left and right eye pictures.

In a scenario where a plurality of users watch a hologram displayed on a sand table display device at the same time, a process of determining the first user among the users may be. For each head mounted display device, a first position of the head mounted display device in a specified environment (i.e., positions of the two eyes of a user wearing the head mounted display device) is determined. And determining a second position of the interactive control pen in the specified environment. Among the head mounted display apparatuses, a head mounted display apparatus whose first position is within a specified distance range (a preset range, which may be 0.5 to 1.2 meters) from the second position is determined as the first standby apparatus. And determining a first included angle corresponding to each first device to be determined. And determining a first to-be-determined device corresponding to the first included angle which is not greater than a third angle threshold (the third angle threshold is a preset value and is greater than the first angle threshold) as a second to-be-determined device. Among the second devices to be determined, the second device to be determined, which is closest to the specified point, is determined as the head mounted display device worn by the first user.

S202: and acquiring the pose information of the interactive control pen.

The pose information of the interactive control pen shows the pointing direction of the interactive control pen. In an optional embodiment of the present specification, the pose information of the interactive control pen includes six-degree-of-freedom information of the interactive control pen, where position 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to image information acquired by a mobile auxiliary camera, rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to data acquired by an IMU module in the interactive control pen, and the mobile auxiliary camera is disposed on the sand table display device.

S204: a first angle between the viewing angle and the pointing direction is determined.

In order to determine the positions of the head-mounted display devices and the interactive control pen, angles such as a first included angle are determined. In an alternative embodiment of the present specification, a three-dimensional coordinate system may be previously established in a specified environment, and the origin of coordinates of the three-dimensional coordinate system is the center of the display screen of the sand-tray type display device.

S206: if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

In this specification, the output frequency of the head-mounted display device is adjusted according to a first included angle between a viewing angle of a first user and a pointing direction of an interaction control pen, and when the first included angle is greater than a first angle threshold, it indicates that a large position difference and/or angle difference exists between partial contents that the first user wants to interact in a hologram, which may cause the first user to be unable to accurately select the partial contents that the first user wants to interact in the hologram. Under the condition that the first included angle is not larger than the first angle threshold, the condition that the first user and the part of content which the first user wants to interact have good interaction is indicated, at the moment, the output frequency of the head-mounted display device is reduced, so that more users can participate in the observation of the holographic image, and the reasonable distribution of holographic image display resources is facilitated.

Now, how to determine the first frequency will be described. In an alternative embodiment of the present specification, the number of users is obtained (the number of users may be obtained in real time to adjust the first frequency in real time). Wherein the users include the first user and a second user, the second user being a user wearing the head mounted display device and not holding the interactive control pen. If the number of the users is larger than a number threshold, adjusting the value of the first frequency to a first value, and if the number of the users is not larger than the number threshold, adjusting the value of the first frequency to a second value, wherein the first value is smaller than the second value. The first value and the second value can be preset values and can be set by a manager of the holographic display system.

Furthermore, the present specification can reduce the consumption of resources while satisfying the viewing requirements of the user, in addition to adjusting the output frequency of the head-mounted display device to reasonably distribute and display resources. In an optional real-time of the present description, the viewpoint information of each second user is acquired (which may be real-time acquisition to adjust the output frequency of the sand table display device in real-time). Wherein the viewpoint information shows a second user dual-purpose position and a second user dual-purpose angle of view, and the second user is a user wearing the head-mounted display device and not holding the interactive control pen. Determining a density center of a distribution of the second user dual-purpose locations in a specified environment, wherein the specified environment is an environment in which the sand table display device is located. And determining a connecting line between the density center and a designated point as a designated line, wherein the designated point is the intersection point of the pointing direction of the interactive control pen and the holographic image. If a second included angle between the direction and the specified line is larger than a second angle threshold, adjusting the output frequency of the sand table type display device to a third frequency, and if the second included angle is not larger than the second angle threshold, adjusting the output frequency of the sand table type display device to a fourth frequency, wherein the third frequency is larger than the fourth frequency.

In practical application scenarios, there may exist a holographic presentation system that only provides a presentation to a user, without interaction with the user. In an optional embodiment of the present specification, in order to determine whether interaction with a user is required, a first pressure sensor is disposed on the head-mounted display device, and when the head-mounted display device is worn on the head of the user, the first pressure sensor generates a first signal and sends the first signal to the display processing device; the interactive control pen is internally provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing equipment; the acquiring of the viewpoint information of the first user includes: and if the first signal is detected at a specified time and the second signal is detected within a specified time range from the specified time, acquiring viewpoint information of the first user.

In the related art, to realize LED multi-view 3D display, shutter-type active three-dimensional glasses with a refresh rate of at least 240Hz are required and support the multi-view display function. The refresh rate of the existing shutter type active three-dimensional glasses in the market is only 120Hz, and the multi-view display function is not supported.

Illustratively, the receiving frequency of the interface chip at the input end of the sending device cannot reach the minimum standard of 240 Hz. The sending device and the 3D display screen are communicated through a network cable, and the transmission frequency can be far higher than 240 Hz. Illustratively, the present specification uses a 4K bandwidth 60Hz refresh rate interface chip, and the input data amount is equivalent to 1080P bandwidth 240 Hz. The sending device in this specification cuts the received 4K pictures into 4 (corresponding to two shutter active three-dimensional glasses) 1080P bandwidth pictures and sequentially transmits the pictures to the LED screen receiving card at a refresh rate of 240Hz or higher for display, which is equivalent to an LED display system capable of displaying 240Hz video stream.

In this specification, multiple shutter active three-dimensional glasses receive pictures from the same 3D display screen, illustratively, a total of 4 pictures matching 2 shutter active three-dimensional glasses are presented in a left 1, left 2, right 1, right 2 order at 4.166ms intervals. The shutter type active three-dimensional glasses support the display refresh rate of 120Hz, but the synchronous signal is a square wave signal, namely a 60Hz square wave signal, a peak half period and a trough half period are contained in a single period, and the state switching is performed 120 times per second, so that the use of the 120Hz display refresh rate of the common shutter type active three-dimensional glasses can be met.

Moreover, the synchronization mode can effectively distinguish the left eye from the right eye, for example, the wave crest is defined as the left eye, and the wave trough is defined as the right eye. Then the frequency of the synchronization signal is 60Hz for a 240Hz 3D image presentation system. Under the synchronous signal of 60Hz, the left eye and the right eye can still be distinguished by utilizing the wave crest and the wave trough.

In an optional embodiment of this specification, the shutter active three-dimensional glasses are shutter active three-dimensional glasses with adjustable signal delay, and the left eye becomes transparent 4.166ms every time a peak signal is received (the peak signal lasts 8.333ms, the transparent state lasts only 4.166ms at this time, and when the peak signal becomes transparent, the picture on the 3D display screen can be received, and when the peak signal does not become transparent, the display area can display black), and the right eye becomes transparent 4.166ms every time a valley is received. The signal delay of one of the glasses is then adjusted to 4.166ms, so that the glasses No. 1 and No. 2 can be distinguished.

The interaction mode is not particularly limited in this specification, and for example, the interaction mode may be to enlarge, reduce, move, rotate, select, and replace the hologram output by the sand table type display device.

In an alternative embodiment of the present disclosure, the sand table display device is an LED (Light Emitting Diode) display device. The image specification corresponding to the first display signal is 1920 pixels by 1080 pixels, and the output frequency of the first display signal is 120Hz, so that the binocular alternating output picture to the user at the frequency of 60Hz is realized.

After receiving the first display signal, the sand-tray type display device outputs a holographic image according to the first display signal, so that a user wearing the head-mounted display device can watch the holographic image.

In order to enable the sand table type display device and the head-mounted type display device to be matched, the display processing equipment generates a synchronous signal according to a left eye first display signal and a right eye first display signal, so that the head-mounted type display device collects a holographic image output by the sand table type display device according to the synchronous signal, and the collected holographic image is displayed to a user wearing the head-mounted type display device in an alternating mode. Therefore, the synchronous signal is used for realizing binocular alternation of output pictures for the user.

In an alternative embodiment of the present description, the head mounted display device is shutter 3D glasses.

Further, when the holographic display system in the present specification provides a holographic display for a plurality of users at the same time, the display processing apparatus in the present specification determines a viewpoint for each head-mounted display device because the viewpoint of each user (i.e., the viewpoint of the head-mounted display device worn by the user) differs to a different degree. Because the number of the head-mounted display devices is not unique, and the first display signals received by the sand table type display device are also not unique, the sand table type display device needs to respectively display the holographic images to different head-mounted display devices according to different first display signals.

Further, in an alternative embodiment of the present disclosure, the hologram displayed on the sand table display device is also matched with the interactive handle to provide a more interactive experience for the user. Specifically, the display processing device judges whether the outline of the holographic image displayed by the sand table type display device intersects with the indicating end of the interactive handle. If not, the indication end of the interactive handle does not point on the holographic image, so that the position of the interactive handle indicated on the holographic image is not clear, the holographic image displayed by the sand table type display device can be enlarged until the outline of the holographic image is intersected with the indication end of the interactive handle.

From the foregoing, it can be seen that the holographic display in this specification requires that the sand table display device and the head mounted display device cooperate. The adaptation is to some extent achieved by means of a synchronization signal. In an alternative embodiment of the present disclosure, all the head-mounted display devices can be controlled by one synchronization signal; in another alternative embodiment of the present disclosure, the synchronization signal may be generated separately for each head-mounted display device to control different head-mounted display devices separately.

Specifically, for a certain head-mounted display device, when the synchronous signal controls the head-mounted display device to display the holographic image under the viewpoint of the head-mounted display device, the content displayed by the sand-disc display device is collected, and the content is displayed to a user according to the collected result.

In an optional embodiment of the present specification, the head-mounted display device collects the holograms output by the sand-tray-type display device according to the synchronization signal, so as to alternately show the collected holograms to a user wearing the head-mounted display device.

The holographic display method in the specification displays the holographic image to a user in a mode that the sand table type display device is matched with the head-mounted display device, the sand table type display device is high in controllability, the displayed brightness can be adjusted according to the brightness of a scene where the sand table type display device is located, and even in a dark scene, a good holographic image display effect can be obtained. Because the holographic image in this description is demonstrateed by the assistance of sand table formula display device, then wear-type display device in this description compares in current VR glasses, need not to carry out comparatively complicated data processing, is favorable to reducing wear-type display device's volume and weight for user's body feels more comfortable.

In addition, the existing products such as 3D display usually only have a narrow viewing viewpoint or can only see the ideal effect at a fixed position, that is, the existing 3D display products only support a single viewpoint, but the 3D stereoscopic interactive display system of single viewpoint display is increasingly unable to meet the needs of customers. In the holographic display method in the specification, a mode that the sand table type display device and the head-mounted display device are matched is adopted, so that the holographic image with the stereoscopic impression can be displayed to the user even if the viewpoint of the user changes (which can be caused by the movement of the user), and the improvement of user experience is facilitated.

In addition, the display processing equipment in the holographic display system controls the display of the sand table type display device according to the viewpoint of the head-mounted display device, and even if the head-mounted display device moves along with the user, the head-mounted display device can collect the holographic images suitable for the user to watch based on the content displayed by the sand table type display device. Furthermore, the holographic display system comprises a plurality of head-mounted display devices, different head-mounted display devices can have different viewpoints when being worn by different users, and by the method in the specification, holographic image display can be simultaneously performed on the plurality of head-mounted display devices with different viewpoints through the same sand table type display device.

In order to flexibly and efficiently realize the control of the sand table type display device, in an optional embodiment of the present specification, the display processing device performs combination processing on the first display signals corresponding to the respective viewpoints after generating the first display signals corresponding to the respective viewpoints; and outputting the composite signal obtained after the combination treatment to the sand table type display device.

In an alternative embodiment of the present disclosure, the composite signal corresponds to an image specification of 1920 pixels by 1080 pixels. The output frequency of the composite signal is 120 x k hertz. Where k is the number of head mounted display devices. When k equals 2, the output frequency of the composite signal is 240 hz.

In order to realize data transmission between the display processing device and the sand table type display device and effectively drive the sand table type display device in cooperation with a multi-viewpoint output scene, in an optional embodiment of the present specification, the holographic display system may further include a receiving card. The display processing device outputs the composite signal to the reception card after generating the composite signal. And the receiving card restores the composite signal into each first display signal and drives the sand table type display device to display the holographic image according to each first display signal.

Specifically, the receiving card outputs each first display signal to the sand table type display device in a time-sharing manner, so that the sand table type display device performs holographic image display according to each first display signal in a time-sharing manner. When the sand table type display device displays the holographic images according to the first display signals corresponding to the ith (i can be any integer and represents any one) head-mounted display device, the ith head-mounted display device collects the holographic images displayed by the sand table type display device according to the synchronous signals, and the collection result is matched with the viewpoint of the ith head-mounted display device.

Therefore, the receiving card in the specification has a certain interface function so as to be matched with a plurality of sand table type display devices to simultaneously display the holographic images, and/or be matched with different types of sand table type display devices to display the holographic images.

By the holographic display method, the holographic image with strong stereoscopic impression and high image quality can be displayed for the user in various scenes with different brightness. In an actual scene, a user may move in a field where the sand-tray-type display device is installed according to his own needs, and there may be a phenomenon that a hologram displayed by the sand-tray-type display device is not matched with an actual viewpoint of the user due to the movement of the user.

In order to provide a hologram with excellent display effect during the moving process of the user and display the hologram at different viewpoints at different times, in an optional embodiment of the present specification, the holographic display system further includes: a camera is captured dynamically. The moving capture camera is arranged on the sand table type display device. The number of the motion capture cameras and the setting positions on the sand table type display device, the shooting angle in the present specification are determined according to the setting placement, shape, size, and the like of the sand table type display device. Specifically, the number of the motion capture cameras is 4, which are respectively provided at the corners of the sand table type display device. The motion capture camera is used for image acquisition (shooting) of the environment around the sand-tray type display device so as to capture the position of the user in the scene and further obtain the viewpoint of the head-mounted display device.

In other implementations of the present disclosure, the motion capture camera may be disposed above the sand-tray display device to capture images of the environment from a bird's-eye view.

The motion capture camera may perform image acquisition under control of the display processing device.

In an alternative embodiment of the present specification, the motion capture camera first acquires 3D coordinate information of the head-mounted display device in the space, and then combines the acquired eye viewpoint coordinate information of the user wearing the head-mounted display device in the motion capture space to complete tracking and positioning of the viewing angle of the user, so as to transmit the positioning information to the display processing device. And synchronizing the position signal of the display processing equipment in the motion capture space to the hologram, and taking the three-dimensional position of the head-mounted display device in the motion capture space as the position of the virtual camera (namely the position of the head-mounted display device) in the hologram to calculate the viewpoint picture of the virtual camera.

Specifically, the display processing device may include: a display server, a display processor and a sync transmitter. The first display signal is generated by a display server; the composite signal and/or the synchronization signal are generated by the display processor; the synchronization signal is sent by the synchronization transmitter to the head mounted display device. Optionally, the aforementioned left eye display signal and right eye display signal are generated by a display server, and the display server sends the left eye display signal and the right eye display signal to the display processor after generating the left eye display signal and the right eye display signal. And the display processor generates a first display signal according to the left eye display signal and the right eye display signal and sends the first display signal to the sand table type display device. The synchronization signal is sent by the synchronization transmitter to the head mounted display device.

Optionally, the display server is electrically connected to the display processor through a DP or HDMI line; the display processor is electrically connected with the receiving card through the network port; the display processor is electrically connected with the synchronous transmitter through a cable. The receiving card is electrically connected with the sand table type display device through a flat cable.

In an alternative embodiment of the present specification, the sand-tray display device may be composed of a small-pitch LED display screen, and the sand-tray display device includes an LED display screen body, a three-dimensional video fusion device, a three-dimensional signal transmitter, and the like.

Wherein, the three-dimensional video fusion device is connected with a sending card of the LED display screen body. The three-dimensional signal transmitter is connected to the three-dimensional video fusion device, the three-dimensional video fusion device receives viewpoint picture information of a virtual camera (namely, a head-mounted display device) processed by a display processor in the display processing equipment, the display frequency is adjusted, and pictures are displayed on the LED display screen body through the transmitting card so as to display left and right eye pictures, and the display processor is loaded with a holographic three-dimensional display algorithm. The three-dimensional signal emitter synchronously sends the display frequency to the holographic 3D glasses so as to realize real-time display of pictures adjusted according to different positions of the viewpoint of a user wearing the holographic 3D glasses.

In the present description, the three-dimensional position of the head-mounted display device in the motion capture space is taken as the position of the virtual camera in the virtual space in the virtual three-dimensional scene, so that the position of the virtual camera can be simulated to capture the picture of the position of the head-mounted display device, and the virtual three-dimensional scene is set according to the position of the virtual camera in the virtual space.

The picture taken by the virtual camera (i.e. the picture from the viewpoint of the head-mounted display device) is the three-dimensional virtual scene picture (i.e. the hologram) seen from the viewpoint of the user wearing the head-mounted display device, i.e. the virtual three-dimensional scene taken by the virtual camera is the 3D viewpoint picture seen by the user. The display processing equipment can process the pictures to be displayed in real time according to the viewpoint position information of the head-mounted display device in the motion capture space, and meanwhile, the viewpoint pictures are displayed by adopting the sand table type display device, so that an excellent holographic three-dimensional display effect can be realized, and an excellent screen-out feeling and holographic visual feeling can be obtained.

Compared with other products, the holographic display system has the advantages that the cost is greatly reduced, the picture display is more exquisite and rich, and the color is more rich. The display effect stability is strong, the user use is more convenient, and the operation control is simpler. The sand table type display device is fine and smooth in display effect, high in brightness and low in cost, and the LED active 3D stereoscopic display technology is combined with the space position information of the observation viewpoint, so that the 3D stereoscopic effect that the virtual object is suspended on the LED display screen is achieved, the user impression can be improved, and the holographic display effect is improved.

In addition, in an alternative embodiment of the present specification, the processes in the present specification can be used not only for displaying the hologram to the user, but also for realizing the interaction with the user. Specifically, when receiving the interaction instruction, the display processing device generates a second display signal and outputs the second display signal to the sand table type display device, so that the sand table type display device outputs a holographic image for displaying the interaction handle according to the second display signal, and a user can operate the interaction handle to realize interaction.

Optionally, the interaction instruction is triggered by a specified action performed by the user through the interactive control pen, which may be captured by the motion capture camera. The holographic image of the interactive handle can be a bar-shaped holographic image, one end of the interactive handle is matched with the hand motion of the user and moves in the holographic image generated by the sand table type display device along with the hand motion of the user.

Based on the same idea, the present specification further provides a holographic display, as shown in fig. 3, comprising one or more of the following modules:

a first obtaining module 300 configured to: acquiring viewpoint information of a first user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the first user, and the first user is a user holding the interactive control pen;

a second obtaining module 302 configured to: acquiring pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;

a first angle determination module 304 configured to: determining a first angle between the viewing angle and the pointing direction;

a first adjustment module 306 configured to: if the first included angle is larger than a first angle threshold, adjusting the output frequency of the head-mounted display device to a first frequency, and if the first included angle is not larger than the first angle threshold, adjusting the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is larger than the second frequency.

In an alternative embodiment of the present description, the holographic display also includes a second adjustment module. The second adjustment module is configured to: acquiring the number of users, wherein the users comprise the first user and a second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen; if the number of the users is larger than a number threshold, adjusting the value of the first frequency to a first value, and if the number of the users is not larger than the number threshold, adjusting the value of the first frequency to a second value, wherein the first value is smaller than the second value.

In an optional embodiment of the present description, the holographic display apparatus further comprises a third adjustment module, the third adjustment module is configured to: acquiring viewpoint information of each second user, wherein the viewpoint information shows a dual-purpose position and a dual-purpose view angle of the second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen; determining a density center of distribution of the second user dual-purpose positions in a specified environment, wherein the specified environment is the environment where the sand table type display device is located; determining a connecting line between the density center and a designated point as a designated line, wherein the designated point is an intersection point of the pointing direction of the interactive control pen and the holographic image; if a second included angle between the direction and the specified line is larger than a second angle threshold, adjusting the output frequency of the sand table type display device to a third frequency, and if the second included angle is not larger than the second angle threshold, adjusting the output frequency of the sand table type display device to a fourth frequency, wherein the third frequency is larger than the fourth frequency.

In an alternative embodiment of the present description, the viewpoint information comprises six-degree-of-freedom information of a head mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm.

In an optional embodiment of the present specification, the pose information of the interactive control pen includes six-degree-of-freedom information of the interactive control pen, where position 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to image information acquired by a mobile auxiliary camera, rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to data acquired by an IMU module in the interactive control pen, and the mobile auxiliary camera is disposed on the sand table display device.

In an alternative embodiment of the present disclosure, the head-mounted display device includes shutter-type active three-dimensional glasses and at least three reflective mark points embedded on a housing structure of the shutter-type active three-dimensional glasses, and the shutter-type active three-dimensional glasses are used for receiving the holographic images transmitted by the sand-tray-type display device, so as to obtain correct left and right eye pictures.

In an optional embodiment of the present specification, a first pressure sensor is disposed on the head-mounted display device, and when the head-mounted display device is worn on the head of a user, the first pressure sensor generates a first signal and sends the first signal to the display processing apparatus; the interactive control pen is internally provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing equipment; the first obtaining module 300 is specifically configured to: and if the first signal is detected at a specified time and the second signal is detected within a specified time range from the specified time, acquiring viewpoint information of the first user.

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 4, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.

And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.

The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the holographic display device and/or the second holographic display device on the logic level. And the processor is used for executing the program stored in the memory and is specifically used for executing any one of the holographic display processes.

It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.