阅读说明：本技术 用于电影院的光场显示系统 (Light field display system for cinema ) 是由 J·S·卡拉夫 B·E·比弗森 J·多姆 于 2020-03-11 设计创作，主要内容包括：一种用于向电影院中的观看者显示全息内容(例如,全息电影或增强电影的全息内容)的光场(LF)显示系统。所述电影院中的所述LF显示系统包含平铺在一起以形成LF模块阵列的LF显示模块。所述LF模块阵列创建了用于在所述电影院中显示所述全息内容的全息对象体。所述LF模块阵列向观看体中的观看者显示所述全息内容。所述LF显示系统可包含在LF电影网络中。所述LF电影网络允许在一个位置创建全息内容且在另一位置呈现全息内容。所述LF电影网络包含管理全息演出内容的数字权限的网络系统。(A Light Field (LF) display system for displaying holographic content (e.g., holographic content of a holographic movie or enhanced movie) to viewers in a movie theater. The LF display system in the theater contains LF display modules tiled together to form an array of LF modules. The array of LF modules creates a holographic object volume for displaying the holographic content in the movie theater. The array of LF modules displays the holographic content to a viewer in a viewing volume. The LF display system may be included in an LF movie network. The LF cinema network allows for the creation of holographic content at one location and the presentation of holographic content at another location. The LF movie network includes a network system that manages digital rights to the holographic presentation content.)

1. A Light Field (LF) display system, comprising:

a processing engine configured to generate holographic content for display by a light field display assembly, the holographic content being displayed as electromagnetic energy;

a light field display assembly, comprising:

one or more of the display surfaces configured to project holographic content; and

one or more of energy devices configured to receive holographic content from the processing engine and generate electromagnetic energy to project the electromagnetic energy through the display surface as holographic content to a viewer.

2. The LF display system of claim 1, wherein the holographic content is presented to the audience in a movie theater.

3. The LF display system of claim 1, wherein the holographic content is presented to one or more viewers in the audience, and the holographic content presented to a first viewer of the one or more viewers is different from the holographic content presented to a second viewer of the one or more viewers.

4. The LF display system of claim 1, wherein the electromagnetic energy is in the visible spectrum.

5. The LF display system of claim 1, wherein the electromagnetic energy is in the infrared spectrum or the ultraviolet spectrum.

6. The LF display system of claim 1, wherein

The processing engine is configured to generate additional content for rendering as a second type of energy by the light field display assembly,

one or more of the display surfaces are configured to project the additional content as the second type of energy, and

one or more of the energy devices are configured to receive the additional content from the processing engine and generate the second type of energy to project the additional content to the viewer through the display surface.

7. The LF display system of claim 6, wherein the second type of energy is acoustic energy that is projected to the viewer as audio content of the holographic content.

8. The LF display system of claim 6, wherein the second type of energy is ultrasonic energy, the ultrasonic energy being presented to the viewer as one or more tactile surfaces.

9. The LF display system of claim 6, wherein the LF display assembly is configured to project the second type of energy as one or more haptic surfaces and project the electromagnetic energy as holographic content, the haptic surfaces and holographic content being projected to the audience at the same convergence point.

10. The LF display system of claim 1, wherein

One or more of the display surfaces are configured to receive a second type of energy, and

one or more of the energy devices are configured to sense the second type of energy.

11. The LF display system of claim 10, wherein the second type of energy is electromagnetic energy.

12. The LF display system of claim 10, wherein the second type of energy is acoustic energy representing audio content.

13. The LF display system of claim 10, wherein the second type of energy is ultrasonic energy.

14. The LF display system of claim 10, wherein the processing engine is configured to record measurements of the second type of energy sensed by one or more of the energy devices.

15. The LF display system of claim 1, wherein the LF display assembly further comprises:

one or more of the energy surfaces comprising a plurality of energy locations, a first subset of the energy locations configured to emit electromagnetic energy generated by the energy device representing holographic content.

16. The LF display system of claim 15, wherein the energy surface comprises:

a second subset of energy locations configured to emit a second type of energy representative of additional content for presentation to the viewer.

17. The LF display system of claim 15, wherein the energy surface comprises:

a second subset of energy locations configured to receive a second type of energy.

18. The LF display system of claim 17, wherein the second type of energy is electromagnetic energy representing visible light content received from a volume in front of the display surface, and the second subset of energy locations is configured to receive visible light content.

19. The LF display system of claim 15, wherein the energy surface comprises:

a second subset of energy locations configured for a second type of energy, the second subset of energy locations interleaved with the first subset of energy locations on the energy surface.

20. The LF display system of claim 1, wherein the LF display assembly further comprises:

a plurality of waveguides configured to receive electromagnetic energy generated by the energy device representing holographic content and project the electromagnetic energy from the display surface toward the viewer along a plurality of propagation paths.

21. The LF display system of claim 20, wherein the LF display assembly further comprises:

a plurality of energy locations configured to emit the electromagnetic energy representing holographic content, and the plurality of waveguides configured to direct the electromagnetic energy from the plurality of energy locations toward the display surface for projection to the viewer, and

wherein each propagation path corresponds to an energy location of the plurality of energy locations, and a direction of each propagation path is based on a location of the energy location from which it was transmitted.

22. The LF display system of claim 20, further comprising:

a second plurality of waveguides configured to receive a second type of energy representing additional content from the energy device and project the second type of energy from the display surface toward the viewer along a second plurality of propagation paths.

23. The LF display system of claim 22, wherein the first plurality of waveguides and the second plurality of waveguides are interleaved.

24. The LF display system of claim 20, further comprising:

a second plurality of waveguides configured to receive a second type of energy from the display surface and to convert the second type of energy toward the energy locations.

25. The LF display system of claim 24, further comprising a plurality of energy locations configured to receive the second type of energy, and wherein:

receiving the second type of energy from a plurality of incident paths at the display surface;

the second plurality of waveguides guides the second type of energy from the display surface to the plurality of energy locations configured to receive the second type of energy, an

Each energy location of the plurality of energy locations configured to receive the second type of energy corresponds to an incident path of the plurality of incident paths.

26. The LF display system of claim 1, further comprising:

one or more of energy relays configured to relay electromagnetic energy generated by the energy device representing holographic content to one or more energy surfaces.

27. The LF display system of claim 26, wherein the one or more energy surfaces are substantially seamless.

28. The LF display system of claim 26, wherein

One or more of the energy repeaters are configured to repeat a second type of energy representing additional content from the energy device to the one or more energy surfaces.

29. The LF display system of claim 28, wherein the energy repeater configured to repeat electromagnetic energy and the energy repeater configured to repeat the second type of energy are interleaved.

30. The LF display system of claim 26, wherein

An energy relay of the energy relays is configured to relay a second type of energy from the energy surface to the energy device.

31.60^*The LF display system of claim 1, wherein the plurality of energy devices includes:

one or more electromagnetic energy devices that generate electromagnetic energy for projection as holographic content to the audience.

32. The LF display system of claim 1, wherein the plurality of energy devices includes:

one or more ultrasonic energy sources configured to generate ultrasonic energy for presentation to the audience as one or more volumetric haptic surfaces.

33. The LF display system of claim 1, the plurality of energy devices comprising:

an electrostatic speaker array coupled to a plurality of waveguide elements, the electrostatic speaker array comprising:

at least one transparent film configured to generate acoustic energy when driven, an

A plurality of electrodes configured to acoustically drive the transparent film, each electrode of the plurality of electrodes located between one or more waveguide elements of the plurality of waveguide elements.

34. The LF display system of claim 1, the plurality of energy devices comprising:

one or more energy sensors configured to sense energy incident on the one or more display surfaces.

35. The LF display system of claim 34, wherein the one or more energy sensors are configured to capture a light field from electromagnetic energy incident on the display surface.

36. The LF display system of claim 35, wherein the LF display assembly is configured to simultaneously project holographic content and capture a light field.

37. The LF display system of claim 1, further comprising:

a tracking system configured to obtain information about a viewer viewing the holographic content.

38. The LF display system of claim 27, wherein the information obtained by the tracking system includes:

viewer response to holographic content, an

Characteristics of a viewer viewing the holographic content.

39. The LF display system of claim 37, wherein the information about the viewer includes any of a location of the viewer, a movement of the viewer, a gesture of the viewer, an expression of the viewer, an age of a viewer, a gender of the viewer, and clothing worn by the viewer.

40. The LF display system of claim 37, wherein the holographic content generated by the processing engine is altered in response to age, gender, preferences, positioning, movement, gestures, or facial expressions of one or more viewers identified by the tracking system.

41. The LF display system of claim 1, further comprising:

a viewer profiling system configured to

Identifying a viewer watching said holographic content presented by the LF display module, an

A viewer profile is generated for each of the identified viewers.

42. The LF display system of claim 41, wherein the viewer profiling system is configured to identify viewer responses to the holographic content or characteristics of viewers viewing the holographic content, and include the identified responses or characteristics in a viewer profile.

43. The LF display system of claim 41, wherein the viewer profiling system accesses social media accounts of the one or more identified viewers to generate a viewer profile.

44. The LF display system of claim 41, wherein the holographic content generated by the processing engine is altered in response to one or more viewer profiles of viewers viewing the holographic content displayed by the LF display assembly.

45. The LF display system of claim 1, wherein the LF processing engine is configured to create the holographic content based in part on one or more identified viewers of the audience, each identified viewer viewing the holographic content displayed by the LF display system and associated with a viewer profile containing one or more characteristics.

46. The method of claim 45, wherein the characteristics include any of a positioning of the viewer, a motion of the viewer, a gesture of the viewer, a facial expression of a user, a gender of the user, an age of the user, and clothing of the user.

47. The LF display system of claim 45, wherein the processing engine further includes:

a processor configured to apply a model to:

identifying a particular viewer of the one or more viewers viewing the displayed holographic content using information obtained by the tracking system,

identifying one or more characteristics of the particular viewer based on the viewer profile of the identified particular user,

determining preferences of the particular viewer based on the identified characteristics, an

Creating holographic content for presentation by the LF display system to the particular viewer in accordance with the determined preferences.

48. The LF display system according to claim 46, wherein the model is a neural network trained with reinforcement learning.

49. The LF display system of claim 46, wherein the holographic content presented by the LF display module is a movie and the created holographic content enhances the movie.

50. The LF display system of claim 1, wherein the LF display assembly further comprises:

a plurality of LF display modules, each module comprising one or more of the energy devices and one or more of the display surfaces, and wherein the plurality of LF display modules form a seamless display surface.

51. The LF display system of claim 50, wherein the surface area of the seamless display surface is greater than the surface area of the display surface of a single LF display module.

52. A Light Field (LF) display system, comprising:

a processing engine configured to generate holographic content for display by a light field display assembly; and

a light field display assembly, comprising:

one or more energy devices configured to generate the holographic content received from the processing engine,

a plurality of display surface locations, each surface location configured to project a portion of the holographic content, the plurality of display surface locations comprising:

a first subset of the display surface locations configured to project a first portion of the holographic content at a deflection angle relative to an original projection angle.

53. The LF display system of claim 52, wherein the deflection angle is a measure of a plurality of projection paths of the portion of holographic content projected from the first subset of display surface locations.

54. The LF display system of claim 52, wherein the deflection angle is an average deflection of a plurality of projection paths of the portion of holographic content projected from the first subset of display surface locations.

55. The LF display system of claim 52, wherein the deflection angle is an intermediate deflection of the projection path of the portion of holographic content projected from the first subset of display surface locations.

56. The LF display system of claim 52, wherein the deflection angle is substantially non-zero for the portion of the holographic content deflected at the deflection angle.

57. The LF display system of claim 52, wherein the deflection angle is an angle of an optical axis of the plurality of projection paths of the portion of holographic content relative to an optical axis of a remaining display surface location where the portion of holographic content is projected.

58. The LF display system of claim 57, wherein the optical axis of each display surface location is an axis of symmetry of a plurality of projection paths of the holographic content projected from the display surface location.

59. The LF display system of claim 52, wherein the original projection angle is a normal to a surface of the light field display assembly.

60. The LF display system of claim 52, wherein the original projection angle is a measure of a plurality of projection paths of the holographic content projected from the plurality of display surface locations other than the first subset of display surface locations.

61. The LF display system of claim 52, wherein the original projection angle is a normal to a display surface of the LF display assembly.

62. The LF display system of claim 52, wherein the original projection angle is an angle of the optical axis of the display surface locations other than the first subset of the display surface locations.

63. The LF display system of claim 52, wherein the deflection angle is based on a positioning of the first subset of display surface locations on the light field display assembly.

64. The LF display system of claim 52, wherein the deflection angle varies substantially continuously across a display surface of the LF display assembly.

65. The LF display system of claim 52, wherein the first portion of holographic content projected by the first subset of display surface locations is projected toward a viewer at a deflection angle.

66. The LF display system of claim 52, wherein

Said first portion of holographic content is projected at said deflection angle relative to said original projection angle such that said first portion of holographic content is viewable from a first field of view, an

The holographic content projected at the original projection angle can be viewed from a second field of view.

67. The LF display system of claim 66, wherein the first field of view and the second field of view are substantially different.

68. The LF display system of claim 1, wherein projecting the first portion of holographic content at the deflection achieves at least one of:

the field of view is greater than the planar field of view;

at least one viewer of the viewers is closer to a seamless display surface than a planar threshold interval; and

at least one holographic object is closer to the viewing volume than a planar threshold proximity.

69. The LF display system of claim 52, wherein the plurality of display surface locations further comprise:

a second subset of the display surface locations configured to project a second portion of the holographic content at an additional deflection angle relative to the original projection angle.

70. The LF display system according to claim 69, wherein

The second subset of display surface locations projects the second portion of holographic content such that the second portion is viewable from a second field of view and the first portion of holographic content is viewable from a first field of view.

71. The LF display system of claim 70, wherein the first field of view and the second field of view are different.

72. The LF display system of claim 70, wherein the first field of view and the second field of view are substantially similar.

73. The LF display system of claim 69, wherein the deflection angle is different from the additional deflection angle.

74. The LF display system of claim 52, further comprising:

a display surface comprising the plurality of display surface locations, the display surface further comprising:

a central surface positioned substantially facing the viewer, the central surface being a vertical plane;

one or more side surfaces positioned adjacent to the central surface at an angle relative to the vertical plane; and is

Wherein the central surface and the one or more side surfaces comprise at least a portion of the display surface.

75. The LF display system of claim 74, wherein the vertical plane has a horizontal axis and a vertical axis, and a side panel is adjacent to a center panel along the horizontal axis.

76. The LF display system according to claim 74, wherein the vertical plane has a horizontal axis and a vertical axis, and the side panel is adjacent to the center panel along the vertical axis.

77. The LF display system of claim 74, wherein the deflection angle is applied to the portion of the holographic content projected from at least one of the central surface and the one or more side surfaces.

78. The method of claim 74, wherein placing the one or more side surfaces at an angle achieves at least one of:

the viewer's field of view is larger than the planar field of view;

at least one viewer in the audience is closer to the seamless display surface than a planar threshold separation; and

at least one holographic object is closer to the viewing volume than a planar threshold proximity.

79. The LF display system of claim 52, further comprising:

an optical system comprising a plurality of optical elements configured to redirect, for the first portion of the holographic content deflected at the deflection angle, the first portion of the holographic content projected at the original projection angle to the deflection angle.

80. The LF display system according to claim 79, wherein the optical system is coupled to the LF display system.

81. The LF display system of claim 1, further comprising:

a plurality of waveguides configured to relay the holographic content generated by the energy device from the plurality of energy surface locations to a display surface.

82. An LF display system, comprising:

a network interface configured to receive holographic content over a network connection, the holographic content for display to a viewer as electromagnetic energy; and

a light field display assembly, comprising:

one or more display surfaces configured to project holographic content; and

one or more energy devices configured to receive holographic content from the network interface and generate the electromagnetic energy to project the electromagnetic energy through the display surface as holographic content to a viewer.

83. The LF display system of claim 82, further comprising:

a decoder configured to decode the holographic content into a format that can be rendered by the LF display assembly.

84. The LF display system of claim 82, further comprising:

a processor storing computer instructions that, when executed, cause the processor to:

receiving, by the network interface from the network connection, holographic content in a first format; and

decoding the holographic data in the first format into holographic content in a second format.

85. The LF display system of claim 84, wherein the first format is a vectorized data format and the second format is a rasterized data format.

86. The LF display system of claim 84, wherein the computer instructions, when executed, further cause the processor to:

determining a hardware configuration of the LF display system; and

decoding the holographic content into the second format based on the hardware configuration.

87. The LF display system of claim 84, wherein the computer instructions, when executed, further cause the processor to:

determining a layout of a movie theater;

decoding the holographic content into the second format based on the layout of the movie theater.

88. The LF display system of claim 84, wherein the computer instructions, when executed, further cause the processor to:

determining a configuration of the LF display assembly, the configuration including any of:

the resolution of the image is determined by the resolution,

the range of the angles is such that,

a field of view, and

a display area.

89. The LF display system of claim 82, further comprising:

a rights management module configured to manage digital rights of holographic content received over the network, the digital rights management module allowing the LF display assembly to project holographic content with digital keys by the rights management module.

90. The LF display system of claim 82, wherein the holographic content is received over the network from a holographic content repository connected to the LF display system.

91. The LF display system of claim 90, wherein the holographic content is received from the holographic content repository in response to the LF display system transmitting a payment for the holographic content to the holographic content repository over the network.

92. The LF display system of claim 82, wherein the holographic content is received from a holographic content generation system configured to generate the holographic content

The performance of the scene is recorded,

converting the recording of the live performance into holographic content, and transmitting the converted holographic content to an LF display system over the network.

Background

The present disclosure relates to presenting movies in movie theaters, and in particular, to light field display systems for displaying movies in movie theaters.

Traditionally, movie theaters have been configured to allow a viewer going to the movie to watch the movie on a two-dimensional cinema screen. Unfortunately, limiting the content of a movie to a two-dimensional cinema screen is a less immersive viewing experience. Even theaters that have been augmented with advanced display technologies (e.g., 3D, augmented reality, etc.) can degrade the viewing experience and leave many deficiencies. For example, in theaters with these systems, additional glasses may need to be worn by the viewer, the viewer may feel uncomfortable with the presentation of the movie, or the quality of the movie may degrade when advanced technology is used. Accordingly, a movie theater configured to enhance the viewing experience of viewers while watching a movie without degrading the overall experience would be beneficial.

Disclosure of Invention

The present disclosure describes a Light Field (LF) display system for displaying holographic content of a movie or enhancing holographic content of a traditional movie in a movie theater. The LF display system includes LF display modules that form surfaces (e.g., walls) in the theater, the LF display modules each having a display area, and the LF display modules tiled together to form a seamless display surface having an effective display area that is larger than the display area of the individual LF display modules. The LF display module displays the holographic content in the holographic object volume such that the holographic content is perceivable by viewers in the movie theater.

The holographic content may be created by an LF movie generation system, by an LF processing engine, or any other system capable of creating holographic content for display in a movie theater. The holographic content may be managed by a network system responsible for managing the digital rights of the holographic content.

The LF display engine renders holographic content in the holographic object volume. The holographic object may be at any location in the movie theater. For example, the holographic viewing volume may be on a viewer's head in a movie theater, on a wall of a movie theater, or "behind" a wall of a movie theater. The LF display system may display certain holographic content to a viewer in one viewing volume while displaying different or additional holographic content to the viewer in another viewing volume. The holographic content may be displayed before, during, and/or after the movie is displayed in the theater.

In some embodiments, the LF display system includes a tracking system and/or a viewer profiling system. The tracking system and viewer profiling system may monitor and store characteristics of viewers in the venue, viewer profiles describing the viewers, and/or viewer responses to holographic content in the venue. The holographic content created for display in the venue may be based on any of the monitored or stored information.

In some embodiments, a user may interact with the holographic content, and the interaction may serve as an input to the LF processing engine. For example, in some embodiments, some or all of the LF display systems contain multiple ultrasonic speakers. The plurality of ultrasonic speakers is configured to generate a tactile surface that is consistent with at least a portion of the holographic content. The tracking system is configured to track user interaction with the holographic object (e.g., through images captured by the imaging sensor of the LF display module and/or some other camera). And the LF display system is configured to provide creation of holographic content based on the interaction.

Drawings

FIG. 1 is a diagram of a light field display module to render holographic objects in accordance with one or more embodiments.

FIG. 2A is a cross-section of a portion of a light field display module in accordance with one or more embodiments.

FIG. 2B is a cross-section of a portion of a light field display module in accordance with one or more embodiments.

FIG. 3A is a perspective view of a light field display module in accordance with one or more embodiments.

Fig. 3B is a cross-sectional view of a light field display module including an interleaved energy relay device in accordance with one or more embodiments.

Fig. 4A is a perspective view of a portion of a light field display system tiled in two dimensions to form a single-sided, seamless surface environment in accordance with one or more embodiments.

FIG. 4B is a perspective view of a portion of a light field display system in a multi-faceted, seamless surface environment in accordance with one or more embodiments.

FIG. 4C is a top view of a light field display system having a polymerization surface in a wing-like configuration according to one or more embodiments.

FIG. 4D is a side view of a light field display system with a polymerization surface in an oblique configuration in accordance with one or more embodiments.

Fig. 4E is a top view of a light field display system having a polymeric surface on a front wall of a room in accordance with one or more embodiments.

Fig. 4F is a side view of an LF display system having a polymeric surface on a front wall of a room in accordance with one or more embodiments.

Fig. 5A is a block diagram of a light field display system in accordance with one or more embodiments.

Fig. 5B shows an example LF movie network 550 in accordance with one or more embodiments.

Fig. 6A is a side view of a theater containing a light field display system in accordance with one or more embodiments.

Fig. 6B is a side view of a theater containing a light field display system in accordance with one or more embodiments.

Fig. 7A is a perspective view of a theater containing a light field display system in accordance with one or more embodiments.

Fig. 7B is a perspective view of a movie theater containing a light field display system displaying holographic content in accordance with one or more embodiments.

Fig. 8 is a perspective view of a movie theater containing a light field display system rendering holographic content in accordance with one or more embodiments.

Fig. 9A is a perspective view of a theater containing a light field display system showing holographic content presented to a set of viewing positions in front of the theater in accordance with one or more embodiments.

Fig. 9B is a perspective view of a theater containing an LF display system showing holographic content presented to a set of viewing locations behind the theater in accordance with one or more embodiments.

Fig. 10 is a flow diagram showing a method for displaying holographic content of a movie within an LF movie distribution network in accordance with one or more embodiments.

The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

Detailed Description

SUMMARY

A Light Field (LF) display system is implemented in a movie theater. The LF display system is configured to display the movie to a viewer. The LF display system includes an LF display assembly configured to present holographic content including one or more holographic objects that will be visible to one or more viewers in a viewing volume of the LF display system. The LF display assembly may form a multi-faceted, seamless surface on some or all of one or more surfaces (e.g., walls) in the theater. Typically, the viewer is a person watching a movie presentation in a movie theater, but can be anyone within the movie theater who can view the holographic content.

The holographic content presented by the LF display system may also be enhanced with other sensory stimuli (e.g., tactile and/or audio). For example, an ultrasound source in an LF display system may project ultrasound pressure waves that create a volume tactile projection. The volume haptic projection provides a haptic surface corresponding to some or all of the projected holographic objects. The holographic content may also contain additional visual content (i.e., 2D or 3D visual content). In embodiments with multiple energy sources (i.e., holographic objects that provide the correct tactile sensations and sensory stimuli at any given point in time), the energy source coordination that enables a cohesive experience is part of the LF system. For example, the LF system may include a controller that coordinates presentation of the holographic content and the tactile surface.

In some embodiments, the LF display system may include elements that enable the system to project at least one type of energy while sensing at least one type of energy. The sensed energy can be used to record how the viewer responds to the holographic content. For example, an LF display system may project both holographic objects for viewing and ultrasound for tactile perception, while simultaneously recording imaging information and other scene analysis for tracking the viewer. As an example, such a system may project a holographic tape during a show in a movie, such that when the tape virtually "lands" on the viewer's head, the tape gives the viewer the illusion that the tape is landing on his head. The LF display system components that perform energy sensing of the environment may be integrated into the display surface, or they may be dedicated sensors separate from the display surface.

In some embodiments, an LF display system includes a plurality of LF display modules forming a surface in a theater. The surface may cover, for example, the walls and/or ceiling of a cinema. The LF display module forming the surface may be configured to project holographic content of the movie. For example, rather than the projector projecting a movie on a theater screen so that viewers in the theater can view the movie, the LF display module can present the movie so that viewers in the theater can view the movie.

An LF display module displaying holographic content of a movie may provide an experience with reality that is difficult to distinguish from reality. For example, when viewing holographic content of a movie using an LF display module, the viewer does not need to wear 3D glasses or any other head-mounted device to view the holographic content. Further, the holographic content presented by the LF display module may be responsive to viewer actions (e.g., gestures, eye movements, etc.), and the viewer may be able to touch the holographic content presented by the LF display module.

Alternatively or additionally, in some embodiments, the LF display module may be configured to present holographic content that enhances the movie. For example, the LF display module may present the holographic content in a movie theater while presenting a movie so that a viewer in the movie theater may perceive both the movie and the enhanced holographic content at the same time. Here, the set of LF display modules simultaneously displaying holographic content and a movie enhances the ambience of the movie theater and provides additional content for the movie. More generally, an LF display system in a movie theater enhances the viewing experience of viewers in the movie theater relative to movie theaters without the LF display system.

The LF display system may be part of an LF movie distribution network. LF movie distribution networks allow recording and/or creating holographic content in one location, encoding the holographic content and transmitting it to a different location. Upon receipt, the holographic content may be decoded and displayed as holographic content to a viewer in a movie theater. The holographic content may be the movie itself or the holographic content of an enhanced movie. The LF movie distribution network allows distribution of holographic content to multiple movie theaters. In some embodiments, the LF display system includes a network system that manages digital rights for movies and/or holographic content.

Light field display system

Fig. 1 is a diagram 100 of a Light Field (LF) display module 110 presenting a holographic object 120 in accordance with one or more embodiments. The LF display module 110 is part of a Light Field (LF) display system. The LF display system uses one or more LF display modules to render holographic content containing at least one holographic object. The LF display system may present holographic content to one or more viewers. In some embodiments, the LF display system may also enhance the holographic content with other sensory content (e.g., touch, audio, smell, temperature, etc.). For example, as discussed below, the projection of focused ultrasound waves may generate an aerial haptic sensation that may simulate the surface of some or all of the holographic objects. The LF display system includes one or more LF display modules 110 and is discussed in detail below with respect to fig. 2 through 5.

LF display module 110 is a holographic display that presents holographic objects (e.g., holographic object 120) to one or more viewers (e.g., viewer 140). LF display module 110 includes an energy device layer (e.g., an emissive electronic display or an acoustic projection device) and an energy waveguide layer (e.g., an array of optical lenses). Additionally, the LF display module 110 may contain an energy relay layer for the purpose of combining multiple energy sources or detectors together to form a single surface. At a high level, the energy device layer generates energy (e.g., holographic content) which is then guided to a region in space using an energy waveguide layer according to one or more four-dimensional (4D) light-field functions. LF display module 110 may also project and/or sense one or more types of energy simultaneously. For example, LF display module 110 may be capable of projecting a holographic image and an ultrasonic tactile surface in a viewing volume while detecting imaging data from the viewing volume. The operation of the LF display module 110 is discussed in detail below with respect to fig. 2 through 3.

LF display module 110 uses one or more 4D light field functions (e.g., derived from plenoptic functions) to generate holographic objects within holographic object volume 160. The holographic object may be three-dimensional (3D), two-dimensional (2D), or some combination thereof. Furthermore, the holographic object may be polychromatic (e.g. full color). The holographic objects may be projected in front of the screen plane, behind the screen plane, or separated by the screen plane. The holographic object 120 may be rendered such that it is perceivable anywhere within the holographic object body 160. The holographic object within holographic object 160 may appear to be floating in space to viewer 140.

Holographic object volume 160 represents the volume in which viewer 140 may perceive a holographic object. The holographic object 160 may extend in front of the surface of the display area 150 (i.e. towards the viewer 140) so that the holographic object may be presented in front of the plane of the display area 150. Additionally, holographic object 160 may extend behind the surface of display area 150 (i.e., away from viewer 140), allowing the holographic object to be rendered as if it were behind the plane of display area 150. In other words, holographic object volume 160 may contain all light rays originating from display region 150 (e.g., projected), and may converge to create a holographic object. Herein, the light rays may converge at a point in front of, at, or behind the display surface. More simply, the holographic object volume 160 encompasses all volumes from which a viewer can perceive a holographic object.

Viewing volume 130 is the volume of space from which holographic objects (e.g., holographic object 120) presented within holographic object volume 160 by the LF display system are fully visible. The holographic object may be rendered in a holographic object volume 160 and viewed in a viewing volume 130 such that the holographic object is indistinguishable from an actual object. Holographic objects are formed by projecting the same light rays generated from the surface of the object when physically present.

In some cases, holographic object 160 and corresponding viewing volume 130 may be relatively small such that they are designed for a single viewer. In other embodiments, as discussed in detail below with respect to, for example, fig. 4, 6, 7, 8, and 9, the LF display module may be enlarged and/or tiled to create larger holographic object volumes and corresponding viewing volumes that may accommodate a wide range of viewers (e.g., 1 to thousands). The LF display modules presented in this disclosure can be constructed such that the entire surface of the LF display contains holographic imaging optics, there are no dead or dead space, and no bezel is required. In these embodiments, the LF display modules may be tiled such that the imaging area is continuous over the seams between the LF display modules and the bond lines between tiled modules are barely detectable using the visual acuity of the eye. It is noted that in some configurations, although not described in detail herein, some portions of the display surface may not contain holographic imaging optics.

The flexible size and/or shape of the viewing body 130 allows a viewer to be unconstrained within the viewing body 130. For example, the viewer 140 may move to different positions within the viewing volume 130 and see different views of the holographic object 120 from corresponding viewing angles. To illustrate, referring to fig. 1, the viewer 140 is positioned at a first location relative to the holographic object 120 such that the holographic object 120 appears to be a frontal view of the dolphin. The viewer 140 can move to other positions relative to the holographic object 120 to see different views of the dolphin. For example, the viewer 140 may move so that he/she sees the left side of a dolphin, the right side of a dolphin, etc., much like the viewer 140 is watching an actual dolphin and changing his/her relative positioning to the actual dolphin to see a different view of the dolphin. In some embodiments, holographic object 120 is visible to all viewers within viewing volume 130, all viewers having an unobstructed (i.e., unobstructed by/by objects/people) line of sight to holographic object 120. These viewers may be unconstrained such that they may move around within the viewing volume to see different perspectives of the holographic object 120. Thus, the LF display system can render the holographic object such that multiple unconstrained viewers can simultaneously see different perspectives of the holographic object in real world space as if the holographic object were physically present.

In contrast, conventional displays (e.g., stereoscopic, virtual reality, augmented reality, or mixed reality) typically require each viewer to wear some external device (e.g., 3-D glasses, near-eye displays, or head-mounted displays) to see the content. Additionally and/or alternatively, conventional displays may require that the viewer be constrained to a particular viewing orientation (e.g., on a chair having a fixed position relative to the display). For example, when viewing an object shown by a stereoscopic display, the viewer will always focus on the display surface, rather than on the object, and the display will always present only two views of the object, which will follow the viewer trying to move around the perceived object, resulting in a perceived distortion of the object. However, with light field displays, viewers of holographic objects presented by LF display systems do not need to wear external devices, nor are they necessarily restricted to specific locations to see the holographic objects. The LF display system presents the holographic object in a manner that is visible to the viewer, much the same way that the viewer can see the physical object, without the need for special goggles, glasses, or head-mounted accessories. Furthermore, the viewer may view the holographic content from any location within the viewing volume.

Notably, the size of potential location receptors for holographic objects within the holographic object volume 160. To increase the size of holographic object 160, the size of display area 150 of LF display module 110 may be increased and/or multiple LF display modules may be tiled together in a manner that forms a seamless display surface. The effective display area of the seamless display surface is larger than the display area of each LF display module. Some embodiments related to tiling LF display modules are discussed below with respect to fig. 4 and 6 through 9. As illustrated in fig. 1, the display area 150 is rectangular, resulting in a holographic object 160 that is pyramidal in shape. In other embodiments, the display area may have some other shape (e.g., hexagonal), which also affects the shape of the corresponding viewing volume.

Additionally, although the discussion above focuses on presenting holographic object 120 within a portion of holographic object 160 located between LF display module 110 and viewer 140, LF display module 110 may additionally present content in holographic object 160 behind the plane of display area 150. For example, LF display module 110 may make display area 150 appear to be the surface of the ocean where holographic object 120 is jumping out. And the displayed content may enable the viewer 140 to view through the displayed surface to see marine life underwater. Furthermore, the LF display system can generate content that moves seamlessly around the holographic object 160, both behind and in front of the plane of the display area 150.

Fig. 2A illustrates a cross-section 200 of a portion of an LF display module 210 in accordance with one or more embodiments. The LF display module 210 may be the LF display module 110. In other embodiments, the LF display module 210 may be another LF display module having a display area with a different shape than the display area 150. In the embodiment shown, the LF display module 210 includes an energy device layer 220, an energy relay layer 230, and an energy waveguide layer 240. Some embodiments of LF display module 210 have different components than those described herein. For example, in some embodiments, LF display module 210 does not include energy relay layer 230. Similarly, functionality may be distributed among components in a different manner than described herein.

The display system described herein presents an energy emission that replicates the energy of a typical surrounding object in the real world. Here, the emitted energy is directed from each coordinate on the display surface towards a particular direction. In other words, the respective coordinates on the display surface serve as the projected locations of the emitted energy. The directed energy from the display surface causes a number of energy rays to converge, which can thus create a holographic object. For example, for visible light, the LF display will project from the projection location very many rays that can converge at any point of the holographic object volume, so from the perspective of a viewer positioned further away than the projected object, the rays appear to come from the surface of a real object positioned in the region of this space. In this way, the LF display generates reflected light rays that exit this object surface from the viewer's perspective. The viewer viewing angle may vary over any given holographic object and the viewer will see different views of the holographic object.

As described herein, energy device layer 220 includes one or more electronic displays (e.g., emissive displays such as OLEDs) and one or more other energy projecting and/or energy receiving devices. One or more electronic displays are configured to display content according to display instructions (e.g., from a controller of the LF display system). One or more electronic displays comprise a plurality of pixels, each pixel having an independently controlled intensity. Many types of commercial displays can be used in LF displays, such as emissive LED and OLED displays.

The energy device layer 220 may also contain one or more acoustic projection devices and/or one or more acoustic receiving devices. The acoustic projection device generates one or more pressure waves that are complementary to the holographic object 250. The generated pressure waves may be, for example, audible, ultrasonic, or some combination thereof. An ultrasonic pressure wave array may be used for volume tactile sensation (e.g., at the surface of holographic object 250). The audible pressure waves are used to provide audio content (e.g., immersive audio) that can complement the holographic object 250. For example, assuming that holographic object 250 is a dolphin, one or more acoustic projection devices may be used to (1) generate a tactile surface juxtaposed to the surface of the dolphin so that a viewer may touch holographic object 250; and (2) provide audio content corresponding to a dolphin sounding such as a click, chirp or squeak. Acoustic receiving devices (e.g., microphones or microphone arrays) may be configured to monitor ultrasonic and/or audible pressure waves within a localized area of LF display module 210.

The energy device layer 220 may also contain one or more imaging sensors. The imaging sensor may be sensitive to light in the visible wavelength band and, in some cases, may be sensitive to light in other wavelength bands (e.g., infrared). The imaging sensor may be, for example, a Complementary Metal Oxide Semiconductor (CMOS) array, a Charge Coupled Device (CCD), an array of photodetectors, some other sensor that captures light, or some combination thereof. The LF display system may use data captured by one or more imaging sensors for locating and tracking the position of the viewer.

In some configurations, the energy relay layer 230 relays energy (e.g., electromagnetic energy, mechanical pressure waves, etc.) between the energy device layer 220 and the energy waveguide layer 240. Energy relay layer 230 includes one or more energy relay elements 260. Each energy relay element comprises a first surface 265 and a second surface 270 and relays energy between the two surfaces. The first surface 265 of each energy relay element may be coupled to one or more energy devices (e.g., an electronic display or an acoustic projection device). The energy relay element may be constructed of, for example, glass, carbon, optical fiber, optical film, plastic, polymer, or some combination thereof. Additionally, in some embodiments, the energy relay elements may adjust the magnification (increase or decrease) of the energy passing between the first surface 265 and the second surface 270. If the repeater provides magnification, the repeater may take the form of an array of bonded cone repeaters, known as cones, where the area of one end of the cone may be substantially larger than the area of the opposite end. The large ends of the cones may be bonded together to form a seamless energy surface 275. One advantage is that spaces are created on the small ends of each cone to accommodate mechanical envelopes of multiple energies, such as the bezel of multiple displays. This additional room allows for energy sources to be placed side-by-side on the small cone side, with the active area of each energy source directing energy into the small cone surface and relaying to the large seamless energy surface. Another advantage of using a cone-shaped relay is that there is no non-imaging dead space on the combined seamless energy surface formed by the large ends of the cone. There is no border or border and therefore the seamless energy surfaces can then be tiled together to form a larger surface with few seams depending on the visual acuity of the eye.

The second surfaces of adjacent energy relay elements come together to form an energy surface 275. In some embodiments, the spacing between the edges of adjacent energy relay elements is less than the minimum perceivable profile defined by the visual acuity of a human eye having vision, e.g., 20/40, such that the energy surface 275 is effectively seamless from the perspective of a viewer 280 within the viewing volume 285.

In some embodiments, the second surfaces of adjacent energy relay elements are fused together with a processing step that may involve one or more of pressure, heat, and chemical reaction, in such a way that no seams exist therebetween. And in still other embodiments, the array of energy relay elements is formed by molding one side of a continuous block of relay material into an array of small cone ends, each energy relay element configured to transmit energy from an energy device attached to a small cone end to a larger area of a single combined surface that is never subdivided.

In some embodiments, one or more of the energy relay elements exhibit energy localization, wherein the energy transfer efficiency in a longitudinal direction substantially perpendicular to surfaces 265 and 270 is much higher than the transfer efficiency in a perpendicular transverse plane, and wherein the energy density is highly localized in this transverse plane as the energy wave propagates between surface 265 and surface 270. This localization of energy allows the energy distribution (e.g., image) to be efficiently relayed between these surfaces without any significant loss of resolution.

Energy waveguiding layer 240 uses waveguiding elements in energy waveguiding layer 240 to guide energy from locations (e.g., coordinates) on energy surface 275 to specific energy propagation paths that enter holographic viewing volume 285 outward from the display surface. The energy propagation path is defined by at least two angular dimensions determined by the energy surface coordinate position relative to the waveguide. The waveguide is associated with spatial 2D coordinates. These four coordinates together form a four-dimensional (4D) energy field. As an example, for electromagnetic energy, waveguide elements in energy waveguide layer 240 direct light from locations on seamless energy surface 275 through viewing volume 285 along different propagation directions. In various examples, light is directed according to a 4D light field function to form holographic object 250 within holographic object volume 255.

Each waveguiding element in the energy waveguiding layer 240 may be, for example, a lenslet comprised of one or more elements. In some configurations, the lenslets may be positive lenses. The positive lens may have a spherical, aspherical or free-form surface profile. Additionally, in some embodiments, some or all of the waveguide elements may contain one or more additional optical components. The additional optical component may be, for example, an energy-suppressing structure such as a baffle, a positive lens, a negative lens, a spherical lens, an aspherical lens, a free-form lens, a liquid crystal lens, a liquid lens, a refractive element, a diffractive element, or some combination thereof. In some embodiments, at least one of the lenslets and/or additional optical components is capable of dynamically adjusting its optical power. For example, the lenslets may be liquid crystal lenses or liquid lenses. Dynamic adjustment of the surface profile of the lenslets and/or at least one additional optical component may provide additional directional control of the light projected from the waveguide element.

In the example shown, the holographic object 255 of the LF display has a boundary formed by ray 256 and ray 257, but may be formed by other rays. Holographic object volume 255 is a continuous volume that extends both in front of (i.e., toward viewer 280) and behind (i.e., away from viewer 280) energy waveguide layer 240. In the illustrated example, rays 256 and 257 that may be perceived by the user are projected from opposite edges of LF display module 210 at a maximum angle relative to a normal to display surface 277, but the rays may be rays of other projections. The rays define the field of view of the display and therefore the boundaries of the holographic viewing volume 285. In some cases, the rays define a holographic viewing volume in which the entire display can be viewed without vignetting (e.g., an ideal viewing volume). As the field of view of the display increases, the convergence of rays 256 and 257 will be closer to the display. Thus, a display with a larger field of view allows the viewer 280 to see the entire display at a closer viewing distance. In addition, rays 256 and 257 may form an ideal holographic object volume. Holographic objects presented in an ideal holographic object volume can be seen anywhere in viewing volume 285.

In some instances, the holographic object may be presented to only a portion of viewing volume 285. In other words, the holographic object volume may be divided into any number of viewing sub-volumes (e.g., viewing sub-volume 290). In addition, the holographic object may be projected to the outside of the holographic object body 255. For example, holographic object 251 is present outside holographic object volume 255. Because holographic object 251 is present outside of holographic object volume 255, it cannot be viewed from every location in viewing volume 285. For example, the holographic object 251 may be visible from a position in the viewing sub-volume 290, but not from a position of the viewer 280.

For example, turn to FIG. 2B to show viewing holographic content from a different viewing sub-volume. Fig. 2B illustrates a cross-section 200 of a portion of an LF display module in accordance with one or more embodiments. The cross-section of fig. 2B is the same as the cross-section of fig. 2A. However, FIG. 2B illustrates a different set of light rays projected from LF display module 210. Rays 256 and 257 still form holographic object 255 and viewing volume 285. However, as shown, the rays projected from the top of LF display module 210 and the rays projected from the bottom of LF display module 210 overlap to form respective viewing subvolumes (e.g., viewing subvolumes 290A, 290B, 290C and 290D) within viewing volume 285. Viewers in a first viewing subvolume (e.g., 290A) may be able to perceive holographic content presented in holographic object volume 255, and viewers in other viewing subvolumes (e.g., 290B, 290C, and 290D) may not be able to perceive.

More simply, as illustrated in fig. 2A, holographic object volume 255 is a volume in which holographic objects may be rendered by an LF display system such that the holographic objects may be perceived by a viewer (e.g., viewer 280) in viewer volume 285. In this way, viewing volume 285 is an example of an ideal viewing volume, and holographic object 255 is an example of an ideal object. However, in various configurations, a viewer may perceive holographic objects presented by LF display system 200 in other example holographic objects. More generally, a "line of sight guide" will be applied when viewing holographic content projected from the LF display module. The gaze guidance asserts that the line formed by the viewer's eye location and the holographic object being viewed must intersect the LF display surface.

Because the holographic content is rendered according to the 4D light field function, each eye of the viewer 280 sees a different viewing angle of the holographic object 250 when viewing the holographic content rendered by the LF display module 210. Furthermore, as viewer 280 moves within viewing volume 285, he/she will also see different viewing angles for holographic object 250, as will other viewers within viewing volume 285. As will be appreciated by those of ordinary skill in the art, 4D light field functions are well known in the art and will not be described in further detail herein.

As described in more detail herein, in some embodiments, the LF display may project more than one type of energy. For example, an LF display may project two types of energy, e.g., mechanical energy and electromagnetic energy. In this configuration, the energy relay layer 230 may contain two separate energy relays that are interleaved together at the energy surface 275, but separated such that energy is relayed to two different energy device layers 220. Here, one repeater may be configured to transmit electromagnetic energy, while another repeater may be configured to transmit mechanical energy. In some embodiments, mechanical energy may be projected from a location on energy waveguide layer 240 between electromagnetic waveguide elements, thereby facilitating formation of a structure that inhibits light from being transmitted from one electromagnetic waveguide element to another. In some embodiments, the energy waveguide layer 240 may also include waveguide elements that transmit focused ultrasound along a particular propagation path according to display instructions from the controller.

Note that in an alternative embodiment (not shown), LF display module 210 does not include energy relay layer 230. In this case, energy surface 275 is an emitting surface formed using one or more adjacent electronic displays within energy device layer 220. And in some embodiments, the spacing between the edges of adjacent electronic displays is less than the minimum perceivable profile defined by the visual acuity of a human eye having 20/40 vision without an energy relay layer, such that the energy surface is effectively seamless from the perspective of a viewer 280 within the viewing volume 285.

LF display module

Fig. 3A is a perspective view of an LF display module 300A in accordance with one or more embodiments. LF display module 300A may be LF display module 110 and/or LF display module 210. In other embodiments, the LF display module 300A may be some other LF display module. In the embodiment shown, the LF display module 300A includes an energy device layer 310 and an energy relay layer 320 and an energy waveguide layer 330. LF display module 300A is configured to render holographic content from display surface 365, as described herein. For convenience, display surface 365 is shown in dashed outline on frame 390 of LF display module 300A, but more precisely the surface directly in front of the waveguide elements defined by the inner edge of frame 390. The display surface 365 includes a plurality of projection locations from which energy may be projected. Some embodiments of LF display module 300A have different components than those described herein. For example, in some embodiments, the LF display module 300A does not include an energy relay layer 320. Similarly, functionality may be distributed among components in a different manner than described herein.

Energy device layer 310 is an embodiment of energy device layer 220. The energy device layer 310 includes four energy devices 340 (three are visible in the figure). The energy devices 340 may all be of the same type (e.g., all electronic displays) or may comprise one or more different types (e.g., comprising an electronic display and at least one acoustic energy device).

Energy relay layer 320 is an embodiment of energy relay layer 230. The energy relay layer 320 includes four energy relay devices 350 (three are visible in the figure). Energy relay devices 350 may all relay the same type of energy (e.g., light) or may relay one or more different types (e.g., light and sound). Each of the relay devices 350 includes a first surface and a second surface, the second surface of the energy relay devices 350 being arranged to form a single seamless energy surface 360. In the illustrated embodiment, each of the energy relays 350 is tapered such that the first surface has a smaller surface area than the second surface, which allows for a mechanical envelope of the energy device 340 to be accommodated on the small end of the taper. This also leaves the seamless energy surface unbounded, since the entire area can project energy. This means that this seamless energy surface can be tiled by placing multiple instances of LF display module 300A together without dead space or borders, so that the entire combined surface is seamless. In other embodiments, the surface areas of the first and second surfaces are the same.

Energy waveguide layer 330 is an embodiment of energy waveguide layer 240. The energy waveguide layer 330 comprises a plurality of waveguide elements 370. As discussed above with respect to fig. 2, the energy waveguide layer 330 is configured to direct energy from the seamless energy surface 360 along a particular propagation path according to a 4D light field function to form a holographic object. Note that in the embodiment shown, the energy waveguide layer 330 is defined by a frame 390. In other embodiments, the frame 390 is not present and/or the thickness of the frame 390 is reduced. The removal or reduction of the thickness of the frame 390 may facilitate tiling the LF display module 300A with additional LF display modules.

Note that in the illustrated embodiment, the seamless energy surface 360 and the energy waveguide layer 330 are planar. In alternative embodiments not shown, the seamless energy surface 360 and the energy waveguide layer 330 may be curved in one or more dimensions.

The LF display module 300A may be configured with additional energy sources residing on the surface of the seamless energy surface and allowing projection of energy fields other than light fields. In one embodiment, the acoustic energy field may be projected from electrostatic speakers (not shown) mounted at any number of locations on the seamless energy surface 360. Further, the electrostatic speaker of the LF display module 300A is positioned within the light field display module 300A such that the dual energy surface simultaneously projects the sound field and the holographic content. For example, an electrostatic speaker may be formed with one or more diaphragm elements that transmit electromagnetic energy at some wavelengths and driven by one or more conductive elements (e.g., a plane that sandwiches the one or more diaphragm elements). The electrostatic speaker may be mounted on the seamless energy surface 360 such that the diaphragm element covers some of the waveguide elements. The conductive electrodes of the speaker may be positioned at the same location as structures designed to inhibit light transmission between electromagnetic waveguides, and/or at locations between electromagnetic waveguide elements (e.g., frame 390). In various configurations, the speaker may project audible sounds and/or generate many sources of focused ultrasound energy for the tactile surface.

In some configurations, the energy device 340 may sense energy. For example, the energy device may be a microphone, a light sensor, a sound transducer, or the like. Thus, the energy relay may also relay energy from the seamless energy surface 360 to the energy device layer 310. That is, the seamless energy surface 360 of the LF display module forms a bi-directional energy surface when the energy device and the energy relay device 340 are configured to simultaneously transmit and sense energy (e.g., transmit a light field and sense sound).

More broadly, the energy device 340 of the LF display module 340 may be an energy source or an energy sensor. LF display module 300A may contain various types of energy devices that act as energy sources and/or energy sensors to facilitate the projection of high quality holographic content to a user. Other sources and/or sensors may include thermal sensors or sources, infrared sensors or sources, image sensors or sources, mechanical energy transducers that generate acoustic energy, feedback sources, and the like. Multiple other sensors or sources are possible. Further, the LF display modules may be tiled such that the LF display modules may form an assembly that projects and senses multiple types of energy from a large aggregate seamless energy surface.

In various embodiments of LF display module 300A, the seamless energy surface 360 may have various surface portions, where each surface portion is configured to project and/or emit a particular type of energy. For example, when the seamless energy surface is a dual energy surface, the seamless energy surface 360 includes one or more surface portions that project electromagnetic energy and one or more other surface portions that project ultrasonic energy. The surface portions of the projected ultrasonic energy may be positioned on the seamless energy surface 360 between the electromagnetic waveguide elements and/or co-located with structures designed to inhibit light transmission between the electromagnetic waveguide elements. In examples where the seamless energy surface is a bi-directional energy surface, the energy relay layer 320 may comprise two types of energy relay devices that are interwoven at the seamless energy surface 360. In various embodiments, the seamless energy surface 360 may be configured such that the portion of the surface below any particular waveguide element 370 is all energy sources, all energy sensors, or a mixture of energy sources and energy sensors.

Fig. 3B is a cross-sectional view of an LF display module 300B containing an interleaved energy relay in accordance with one or more embodiments. Energy relay device 350A transfers energy between energy relay first surface 345A connected to energy device 340A and seamless energy surface 360. Energy relay 350B transfers energy between energy relay first surface 345B connected to energy device 340B and seamless energy surface 360. The two relays are interleaved at an interleaved energy relay 352 connected to a seamless energy surface 360. In this configuration, the surface 360 contains interleaved energy locations of both energy devices 340A and 340B, which may be energy sources or energy sensors. Thus, the LF display module 300B may be configured as a dual energy projection device for projecting more than one type of energy, or as a bi-directional energy device for projecting one type of energy and sensing another type of energy simultaneously. LF display module 300B may be LF display module 110 and/or LF display module 210. In other embodiments, the LF display module 300B may be some other LF display module.

The LF display module 300B contains many components configured similarly to the components of the LF display module 300A in fig. 3A. For example, in the illustrated embodiment, the LF display module 300B includes an energy device layer 310, an energy relay layer 320, a seamless energy surface 360, and an energy waveguide layer 330, including at least the same functions as described with respect to fig. 3A. Additionally, LF display module 300B may present and/or receive energy from display surface 365. Notably, the components of LF display module 300B may be alternatively connected and/or oriented as compared to the components of LF display module 300A in fig. 3A. Some embodiments of LF display module 300B have different components than those described herein. Similarly, functionality may be distributed among components in a different manner than described herein. Fig. 3B illustrates a design of a single LF display module 300B that can be tiled to produce a dual-energy projection surface or bi-directional energy surface with a larger area.

In one embodiment, the LF display module 300B is an LF display module of a bi-directional LF display system. A bi-directional LF display system can simultaneously project energy from the display surface 365 and sense the energy. The seamless energy surface 360 contains both energy projection locations and energy sensing locations that are closely interleaved on the seamless energy surface 360. Thus, in the example of fig. 3B, energy relay layer 320 is configured differently than the energy relay layer of fig. 3A. For convenience, the energy relay layer of the LF display module 300B will be referred to herein as an "interleaved energy relay layer.

The interleaved energy relay layer 320 contains two legs: a first energy relay 350A and a second energy relay 350B. In fig. 3B, each of the legs is shown as a lightly shaded area. Each of the legs may be made of a flexible relay material and formed with sufficient length to be used with various sizes and shapes of energy devices. In some areas of the interleaved energy relay layer, the two legs are tightly interleaved together as they approach the seamless energy surface 360. In the illustrated example, interleaved energy relay 352 is illustrated as a dark shaded area.

When interleaved at the seamless energy surface 360, the energy relay device is configured to relay energy to/from different energy devices. The energy devices are located at the energy device layer 310. As illustrated, energy device 340A is connected to energy relay 350A and energy device 340B is connected to energy relay 350B. In various embodiments, each energy device may be an energy source or an energy sensor.

The energy waveguide layer 330 includes waveguide elements 370 to guide energy waves from the seamless energy surface 360 along a projected path toward a series of convergence points. In this example, holographic object 380 is formed at a series of convergence points. Notably, as illustrated, the convergence of energy at holographic object 380 occurs at the viewer side (i.e., front side) of display surface 365. However, in other examples, the convergence of energy may extend anywhere in the holographic object volume, both in front of display surface 365 and behind display surface 365. The waveguide element 370 can simultaneously guide incoming energy to an energy device (e.g., an energy sensor), as described below.

In one example embodiment of the LF display module 300B, the emissive display is used as an energy source (e.g., energy device 340A) and the imaging sensor is used as an energy sensor (e.g., energy device 340B). In this way, LF display module 300B can simultaneously project holographic content and detect light from a volume in front of display surface 365. In this way, this embodiment of the LF display module 300B functions as both an LF display and an LF sensor.

In one embodiment, the LF display module 300B is configured to simultaneously project a light field from a projection location on the display surface in front of the display surface and capture the light field from in front of the display surface at the projection location. In this embodiment, energy relay 350A connects a first set of locations at seamless energy surface 360 positioned below waveguide element 370 to energy device 340A. In one example, the energy device 340A is an emissive display having an array of source pixels. The energy relay device 340B connects a second set of locations at the seamless energy surface 360 positioned below the waveguide element 370 to the energy device 340B. In one example, energy device 340B is an imaging sensor having an array of sensor pixels. The LF display module 300B may be configured such that the locations at the seamless energy surface 365 below a particular waveguide element 370 are all emission display locations, all imaging sensor locations, or some combination of these locations. In other embodiments, the bi-directional energy surface may project and receive various other forms of energy.

In another example embodiment of the LF display module 300B, the LF display module is configured to project two different types of energy. For example, in one embodiment, energy device 340A is a transmission display configured to transmit electromagnetic energy, and energy device 340B is an ultrasound transducer configured to transmit mechanical energy. Thus, both light and sound may be projected from various locations at the seamless energy surface 360. In this configuration, the energy relay 350A connects the energy device 340A to the seamless energy surface 360 and relays the electromagnetic energy. The energy relay device is configured to have characteristics (e.g., varying refractive index) that enable the energy relay device to efficiently transmit electromagnetic energy. Energy relay 350B connects energy device 340B to seamless energy surface 360 and relays mechanical energy. Energy relay 350B is configured to have features (e.g., distribution of materials having different acoustic impedances) for efficient transmission of ultrasonic energy. In some embodiments, the mechanical energy may be projected from a location between waveguide elements 370 on the energy waveguide layer 330. The location of the projected mechanical energy may form a structure for inhibiting light transmission from one electromagnetic waveguide element to another electromagnetic waveguide element. In one example, a spatially separated array of locations projecting ultrasonic mechanical energy may be configured to create three-dimensional haptic shapes and surfaces in air. The surface may coincide with a projected holographic object (e.g., holographic object 380). In some instances, phase delays and amplitude variations across the array may help create haptic shapes.

In various embodiments, the LF display module 300B with interleaved energy relay devices may contain multiple energy device layers, where each energy device layer contains a particular type of energy device. In these examples, the energy relay layer is configured to relay the appropriate type of energy between the seamless energy surface 360 and the energy device layer 310.

Tiled LF display module

Fig. 4A is a perspective view of a portion of an LF display system 400 tiled in two dimensions to form a single-sided seamless surface environment in accordance with one or more embodiments. LF display system 400 includes a plurality of LF display modules tiled to form an array 410. More specifically, each of the tiles in array 410 represents a tiled LF display module 412. The LF display module 412 may be the same as the LF display module 300A or 300B. The array 410 may cover, for example, some or all of a surface (e.g., a wall) of a room. The LF array may cover other surfaces such as table tops, billboards, round buildings, etc.

The array 410 may project one or more holographic objects. For example, in the illustrated embodiment, array 410 projects holographic object 420 and holographic object 422. Tiling of LF display modules 412 allows for a larger viewing volume and allows objects to be projected at a greater distance from array 410. For example, in the illustrated embodiment, the viewing volume is substantially the entire area in front of and behind array 410, rather than a partial volume in front of (and behind) LF display module 412.

In some embodiments, LF display system 400 presents holographic object 420 to viewer 430 and viewer 434. Viewer 430 and viewer 434 receive different viewing angles for holographic object 420. For example, viewer 430 is presented with a direct view of holographic object 420, while viewer 434 is presented with a more oblique view of holographic object 420. As viewer 430 and/or viewer 434 moves, they are presented with different perspectives of holographic object 420. This allows the viewer to visually interact with the holographic object by moving relative to the holographic object. For example, as viewer 430 walks around holographic object 420, viewer 430 sees different sides of holographic object 420 as long as holographic object 420 remains in the holographic object volume of array 410. Thus, viewer 430 and viewer 434 may simultaneously see holographic object 420 in real world space as if the holographic object were actually present. In addition, viewer 430 and viewer 434 do not need to wear an external device in order to view holographic object 420, because holographic object 420 is visible to the viewer in much the same way that a physical object would be visible. Further, here, holographic object 422 is shown behind the array, as the viewing volume of the array extends behind the surface of the array. In this way, holographic object 422 may be presented to viewer 430 and/or viewer 434.

In some embodiments, the LF display system 400 may include a tracking system that tracks the location of the viewer 430 and the viewer 434. In some embodiments, the tracked location is the location of the viewer. In other embodiments, the tracked location is a location of the viewer's eyes. Eye location tracking is different from gaze tracking, which tracks where the eye is looking (e.g., using orientation to determine gaze location). The eyes of viewer 430 and the eyes of viewer 434 are located at different positions.

In various configurations, the LF display system 400 may include one or more tracking systems. For example, in the illustrated embodiment of fig. 4A, the LF display system includes a tracking system 440 external to the array 410. Here, the tracking system may be a camera system coupled to the array 410. The external tracking system is described in more detail with respect to FIG. 5A. In other example embodiments, the tracking system may be incorporated into the array 410 as described herein. For example, an energy device (e.g., energy device 340) of one or more LF display modules 412 containing a bi-directional energy surface included in the array 410 may be configured to capture an image of a viewer in front of the array 410. In any case, one or more tracking systems of LF display system 400 determine tracking information about viewers (e.g., viewer 430 and/or viewer 434) viewing holographic content presented by array 410.

The tracking information describes a location of the viewer or a location of a portion of the viewer (e.g., one or both eyes of the viewer, or limbs of the viewer) in space (e.g., relative to the tracking system). The tracking system may use any number of depth determination techniques to determine tracking information. The depth determination technique may include, for example, structured light, time-of-flight, stereo imaging, some other depth determination technique, or some combination thereof. The tracking system may include various systems configured to determine tracking information. For example, the tracking system may include one or more infrared sources (e.g., structured light sources), one or more imaging sensors (e.g., red-blue-green-infrared cameras) that may capture infrared images, and a processor that executes a tracking algorithm. The tracking system may use depth estimation techniques to determine the location of the viewer. In some embodiments, LF display system 400 generates holographic objects based on tracked positioning, motion, or gestures of viewer 430 and/or viewer 434 as described herein. For example, LF display system 400 may generate holographic objects in response to a viewer coming within a threshold distance and/or a particular location of array 410.

LF display system 400 may present one or more holographic objects tailored to each viewer based in part on the tracking information. For example, holographic object 420 may be presented to viewer 430 instead of holographic object 422. Similarly, holographic object 422 may be presented to viewer 434 instead of holographic object 420. For example, LF display system 400 tracks the location of each of viewer 430 and viewer 434. LF display system 400 determines a viewing angle for a holographic object that should be visible to a viewer based on the viewer's positioning relative to where the holographic object is to be rendered. LF display system 400 selectively projects light from particular pixels corresponding to the determined viewing angle. Thus, viewer 434 and viewer 430 may have potentially disparate experiences at the same time. In other words, LF display system 400 can present holographic content to a viewing subvolume of the viewing volume (i.e., similar to viewing subvolumes 290A, 290B, 290C and 290D shown in fig. 2B). For example, as illustrated, because LF display system 400 may track the location of viewer 430, LF display system 400 may present spatial content (e.g., holographic object 420) to the viewing subvolumes that surround viewer 430 and wild zoo content (e.g., holographic object 422) to the viewing subvolumes that surround viewer 434. In contrast, conventional systems would have to use separate headphones to provide a similar experience.

In some embodiments, LF display system 400 may include one or more sensory feedback systems. The sensory feedback system provides other sensory stimuli (e.g., tactile, audio, or scent) that enhance holographic objects 420 and 422. For example, in the illustrated embodiment of fig. 4A, LF display system 400 includes a sensory feedback system 442 external to array 410. In one example, the sensory feedback system 442 can be an electrostatic speaker coupled to the array 410. The external sensory feedback system is described in more detail with respect to fig. 5A. In other example embodiments, a sensory feedback system may be incorporated into the array 410, as described herein. For example, the energy device (e.g., energy device 340A in fig. 3B) of the LF display module 412 included in the array 410 may be configured to project ultrasound energy to and/or receive imaging information from a viewer in front of the array. In any case, sensory feedback system presents sensory content to and/or receives sensory content from a viewer (e.g., viewer 430 and/or viewer 434) viewing holographic content (e.g., holographic object 420 and/or holographic object 422) presented by array 410.

LF display system 400 may include a sensory feedback system 442 including one or more acoustic projection devices external to the array. Alternatively or additionally, LF display system 400 may include one or more acoustic projection devices integrated into array 410, as described herein. The acoustic projection device may be comprised of an array of ultrasound sources configured to project a volumetric tactile surface. In some embodiments, for one or more surfaces of the holographic object, the tactile surface may coincide with the holographic object (e.g., at a surface of the holographic object 420) if a portion of the viewer is within a threshold distance of the one or more surfaces. The volume haptic sensation may allow a user to touch and feel the surface of the holographic object. The plurality of acoustic projection devices may also project audible pressure waves that provide audio content (e.g., immersive audio) to the viewer. Thus, the ultrasonic pressure waves and/or the audible pressure waves may act to complement the holographic object.

In various embodiments, the LF display system 400 may provide other sensory stimuli based in part on the tracked location of the viewer. For example, holographic object 422 illustrated in fig. 4A is a lion, and LF display system 400 may cause holographic object 422 to both growl visually (i.e., holographic object 422 appears to be growl) and aurally (i.e., one or more acoustic projection devices project pressure waves) so that viewer 430 perceives it as a growl of the lion from holographic object 422.

It should be noted that in the illustrated configuration, the holographic viewing volume may be limited in a manner similar to viewing volume 285 of LF display system 200 in fig. 2. This may limit the perceived immersion that a viewer will experience with a single wall display unit. One way to address this problem is to use multiple LF display modules tiled along multiple sides, as described below with respect to fig. 4B-4F.

Fig. 4B is a perspective view of a portion of LF display system 402 in a multi-faceted, seamless surface environment in accordance with one or more embodiments. LF display system 402 is substantially similar to LF display system 400 except that multiple LF display modules are tiled to create a multi-faceted seamless surface environment. More specifically, the LF display modules are tiled to form an array that is a six-sided polymeric seamless surface environment. In fig. 4B, multiple LF display modules cover all the walls, ceiling and floor of the room. In other embodiments, multiple LF display modules may cover some, but not all, of the walls, floor, ceiling, or some combination thereof. In other embodiments, multiple LF display modules are tiled to form some other aggregate seamless surface. For example, the walls may be curved such that a cylindrical polymeric energy environment is formed. Further, as described below with respect to fig. 6-9, in some embodiments, the LF display modules may be tiled to form a surface in a movie theater (e.g., wall, etc.).

LF display system 402 may project one or more holographic objects. For example, in the illustrated embodiment, LF display system 402 projects holographic object 420 into an area surrounded by a six-sided polymeric seamless surface environment. In this example, the view volume of the LF display system is also contained within a six-sided polymeric seamless surface environment. Note that in the illustrated configuration, viewer 434 may be positioned between holographic object 420 and LF display module 414, which projects energy (e.g., light and/or pressure waves) used to form holographic object 420. Thus, the positioning of viewer 434 may prevent viewer 430 from perceiving holographic object 420 formed by energy from LF display module 414. However, in the illustrated configuration, there is at least one other LF display module, e.g., LF display module 416, that is unobstructed (e.g., by viewer 434) and that can project energy to form holographic object 420 and be observed by viewer 430. In this way, occlusion by the viewer in space may result in some parts of the holographic projection disappearing, but this effect is much less than if only one side of the volume were filled with the holographic display panel. Holographic object 422 is shown as "outside" the walls of a six-sided polymeric seamless surface environment, since the holographic object body extends behind the polymeric surface. Accordingly, viewer 430 and/or viewer 434 may perceive holographic object 422 as "outside" of the enclosed six-sided environment in which it may move throughout.

As described above with reference to fig. 4A, in some embodiments, LF display system 402 actively tracks the location of the viewer and may dynamically instruct different LF display modules to render holographic content based on the tracked location. Thus, the multi-faceted configuration may provide a more robust environment (e.g., relative to fig. 4A) to provide a holographic object in which an unconstrained viewer may freely move throughout the area encompassed by the multi-faceted seamless surface environment.

It is noted that various LF display systems may have different configurations. Further, each configuration may have a particular orientation of surfaces that converge to form a seamless display surface ("polymerization surface"). In other words, the LF display modules of the LF display system may be tiled to form various aggregation surfaces. For example, in fig. 4B, LF display system 402 contains LF display modules tiled to form six-sided polymeric surfaces that approximate the walls of a room. In some other examples, the polymerized surface may be present on only a portion of the surface (e.g., half of the wall) rather than the entire surface (e.g., the entire wall). Some examples are described herein.

In some configurations, the polymeric surface of the LF display system may include a polymeric surface configured to project energy toward the local viewer. Projecting energy to a local viewing volume allows for a higher quality viewing experience by, for example: increasing the density of projected energy in a particular viewing volume increases the FOV of the viewer in that viewing volume and brings the viewing volume closer to the display surface.

For example, fig. 4C illustrates a top view of LF display system 450A with a polymeric surface in a "winged" configuration. In this example, the LF display system 450A is positioned in a room having a front wall 452, a rear wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown). The first side wall 456, the second side wall 458, the rear wall 454, the floor, and the ceiling are all orthogonal. LF display system 450A includes LF display modules tiled to form a polymeric surface 460 covering the front wall. The front wall 452, and thus the converging surface 460, comprises three portions: (i) a first portion 462 that is substantially parallel to the back wall 454 (i.e., the center surface), (ii) a second portion 464 that connects the first portion 462 to the first side wall 456 and is angled to project energy toward the center of the room (i.e., the first side surface), and (iii) a third portion 466 that connects the first portion 462 to the second side wall 458 and is angled to project energy toward the center of the room (i.e., the second side surface). The first part is a vertical plane in the room and has a horizontal axis and a vertical axis. The second portion and the third portion are angled along a horizontal axis toward the center of the room.

In this example, the viewing volume 468A of LF display system 450A is located in the center of the room and is partially surrounded by three portions of the aggregation surface 460. An aggregated surface at least partially surrounding a viewer ("surrounding surface") increases the immersive experience for the viewer.

For illustration, consider, for example, a polymeric surface having only a central surface. Referring to FIG. 2A, rays projected from either end of the display surface create an ideal hologram and an ideal viewing volume, as described above. Now consider, for example, whether the central surface includes two side surfaces angled toward the viewer. In this case, rays 256 and 257 will be projected at a greater angle from the normal to the central surface. Thus, the field of view of the viewing volume will increase. Similarly, the holographic viewing volume will be closer to the display surface. In addition, since the two second and third portions are tilted closer to the viewing volume, the holographic object projected at a fixed distance from the display surface is closer to the viewing volume.

For simplicity, a display surface with only a central surface has a planar field of view, a planar threshold spacing between the (central) display surface and the viewing volume, and a planar proximity between the holographic object and the viewing volume. The addition of one or more side surfaces angled toward the viewer increases the field of view relative to a planar field of view, decreases the separation between the display surface and the viewing volume relative to a planar separation, and increases the proximity between the display surface and the holographic object relative to a planar proximity. Further angling the side surfaces toward the viewer further increases the field of view, reduces the separation and increases proximity. In other words, the angled placement of the side surfaces increases the immersive experience for the viewer.

In addition, as described below with respect to fig. 6, the deflection optics may be used to optimize the size and positioning of the viewing volume for LF display parameters (e.g., size and FOV).

Returning to fig. 4D, in a similar example, fig. 4D shows a side view of an LF display system 450B with a polymeric surface in a "tilted" configuration. In this example, the LF display system 450B is positioned in a room having a front wall 452, a rear wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. The first side wall, second side wall, rear wall 454, floor 474, and ceiling 472 are all orthogonal. LF display system 450B includes LF display modules tiled to form a polymeric surface 460 covering the front wall. The front wall 452, and thus the converging surface 460, comprises three portions: (i) a first portion 462 that is substantially parallel to the rear wall 454 (i.e., the center surface), (ii) a second portion 464 that connects the first portion 462 to the ceiling 472 and is angled to project energy toward the center of the room (i.e., the first side surface), and (iii) a third portion 464 that connects the first portion 462 to the floor 474 and is angled to project energy toward the center of the room (i.e., the second side surface). The first part is a vertical plane in the room and has a horizontal axis and a vertical axis. The second and third sections are angled toward the center of the room along a vertical axis.

In this example, the viewing volume 468B of the LF display system 450B is located in the center of the room and is partially surrounded by three portions of the aggregation surface 460. Similar to the configuration shown in fig. 4C, the two side portions (e.g., second portion 464 and third portion 466) are angled to enclose the viewer and form an enclosure surface. The enclosing surface increases the viewing FOV from the perspective of any viewer in the holographic viewing volume 468B. In addition, the enclosing surface allows the viewing volume 468B to be closer to the surface of the display, so that the projected objects appear closer. In other words, the angled placement of the side surfaces increases the field of view, reduces the spacing, and increases the proximity of the converging surfaces, thereby increasing the immersive experience for the viewer. Further, as will be discussed below, the deflection optics may be used to optimize the size and positioning of the viewing body 468B.

The angled configuration of the side portions of the polymerized surface 460 enables holographic content to be presented closer to the viewing volume 468B than if the third portion 466 were not angled. For example, the lower extremities (e.g., legs) of a character presented from an LF display system in an inclined configuration may appear closer and more realistic than if an LF display system with a flat front wall were used.

In addition, the configuration of the LF display system and the environment in which it is located may inform the viewing volume and the shape and location of the viewing subvolume.

Fig. 4E, for example, illustrates a top view of an LF display system 450C with a converging surface 460 on a front wall 452 of the room. In this example, the LF display system 450D is positioned in a room having a front wall 452, a rear wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown).

LF display system 450C projects various rays from the converging surface 460. The light rays are projected from each location on the display surface into a range of angles centered on the viewing volume. Rays projected from the left side of the converging surface 460 have a horizontal angular extent 481, rays projected from the right side of the converging surface have a horizontal angular extent 482, and rays projected from the center of the converging surface 460 have a horizontal angular extent 483. Between these points, the projected ray may take on the middle of the range of angles as described below with respect to fig. 6. Having a gradient deflection angle in the projected rays across the display surface in this manner creates a viewing volume 468C. Furthermore, this configuration avoids wasting the resolution of the display when projecting rays into the sidewalls 456 and 458.

Fig. 4F illustrates a side view of an LF display system 450D with a converging surface 460 on a front wall 452 of the room. In this example, the LF display system 450E is positioned in a room having a front wall 452, a rear wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. In this example, the floor is layered such that each layer is stepped up from the front wall to the rear wall. Here, each layer of the floor includes a viewing subvolume (e.g., viewing subvolumes 470A and 470B). The layered floor allows viewing sub-volumes that do not overlap. In other words, each viewing subvolume has a line of sight from the viewing subvolume to the converging surface 460 that does not pass through another viewing subvolume. In other words, this orientation creates a "stadium seating" effect, wherein the vertical offset between the layers allows each layer to "see" the viewing subvolumes of the other layers. An LF display system comprising non-overlapping viewing sub-volumes may provide a higher viewing experience than an LF display system with truly overlapping viewing volumes. For example, in the configuration shown in fig. 4F, different holographic content may be projected to viewers in the viewing sub-volumes 470A and 470B.

Control of LF display system

Fig. 5A is a block diagram of an LF display system 500 in accordance with one or more embodiments. LF display system 500 includes LF display assembly 510 and controller 520. LF display assembly 510 includes one or more LF display modules 512 that project a light field. LF display module 512 may include a source/sensor system 514 that includes one or more integrated energy sources and/or one or more energy sensors that project and/or sense other types of energy. Controller 520 includes data storage 522, network interface 524, and LF processing engine 530. The controller 520 may also include a tracking module 526 and a viewer profiling module 528. In some embodiments, LF display system 500 also includes a sensory feedback system 570 and a tracking system 580. The LF display system described in the context of fig. 1, 2, 3 and 4 is an embodiment of the LF display system 500. In other embodiments, the LF display system 500 includes additional or fewer modules than those described herein. Similarly, functionality may be distributed among modules and/or different entities in a manner different from that described herein. The application of the LF display system 500 will also be discussed in detail below with respect to fig. 6 through 10.

LF display assembly 510 provides holographic content in a holographic object volume that may be visible to a viewer positioned within the viewing volume. LF display assembly 510 may provide holographic content by executing display instructions received from controller 520. The holographic content may include one or more holographic objects projected in front of the aggregation surface of LF display assembly 510, behind the aggregation surface of LF display assembly 510, or some combination thereof. The generation of display instructions with controller 520 is described in more detail below.

LF display assembly 510 provides holographic content using one or more LF display modules included in LF display assembly 510 (e.g., any of LF display module 110, LF display system 200, and LF display module 300). For convenience, one or more LF display modules may be described herein as LF display module 512. LF display modules 512 may be tiled to form LF display assembly 510. The LF display module 512 can be structured into various seamless surface environments (e.g., single sided, multi-sided, cinema wall, curved surface, etc.). In other words, tiled LF display modules form a polymerization surface. As previously described, LF display module 512 includes an energy device layer (e.g., energy device layer 220) and an energy waveguide layer (e.g., energy waveguide layer 240) that render holographic content. LF display module 512 may also include an energy relay layer (e.g., energy relay layer 230) that transfers energy between the energy device layer and the energy waveguide layer when rendering holographic content.

LF display module 512 may also contain other integrated systems configured for energy projection and/or energy sensing as previously described. For example, the light field display module 512 may include any number of energy devices (e.g., energy device 340) configured to project and/or sense energy. For convenience, the integrated energy projection system and the integrated energy sensing system of the LF display module 512 may be collectively described herein as a source/sensor system 514. The source/sensor system 514 is integrated within the LF display module 512 such that the source/sensor system 514 shares the same seamless energy surface with the LF display module 512. In other words, the polymeric surface of LF display assembly 510 includes the functionality of both LF display module 512 and source/sensor module 514. In other words, LF assembly 510 including LF display module 512 with source/sensor system 514 may project energy and/or sense energy while projecting a light field. For example, LF display assembly 510 may include LF display module 512 and source/sensor system 514 configured as a dual energy surface or a bi-directional energy surface as previously described.

In some embodiments, LF display system 500 enhances the generated holographic content with other sensory content (e.g., coordinated touches, audio, or smells) using sensory feedback system 570. Sensory feedback system 570 may enhance the projection of holographic content by executing display instructions received from controller 520. In general, sensory feedback system 570 includes any number of sensory feedback devices (e.g., sensory feedback system 442) external to LF display assembly 510. Some example sensory feedback devices may include coordinated acoustic projection and reception devices, fragrance projection devices, temperature adjustment devices, force actuation devices, pressure sensors, transducers, and the like. In some cases, sensory feedback system 570 may have similar functionality as light field display assembly 510, and vice versa. For example, both the sensory feedback system 570 and the light field display assembly 510 can be configured to generate a sound field. As another example, the sensory feedback system 570 can be configured to generate a tactile surface without the light field display 510 assembly.

To illustrate, in an example embodiment of the light field display system 500, the sensory feedback system 570 may comprise one or more acoustic projection devices. The one or more acoustic projection devices are configured to generate one or more pressure waves complementary to the holographic content upon execution of the display instructions received from the controller 520. The generated pressure waves may be, for example, audible (for sound), ultrasonic (for touch), or some combination thereof. Similarly, sensory feedback system 570 may comprise a fragrance projection device. The fragrance projection arrangement may be configured to provide fragrance to some or all of the target area when executing display instructions received from the controller. The fragrance means may be connected to an air circulation system (e.g. a duct, fan or vent) to coordinate the air flow within the target area. In addition, sensory feedback system 570 may include a temperature adjustment device. The temperature adjustment device is configured to increase or decrease the temperature in some or all of the target zones when executing display instructions received from the controller 520.

In some embodiments, sensory feedback system 570 is configured to receive input from a viewer of LF display system 500. In this case, the sensory feedback system 570 includes various sensory feedback devices for receiving input from a viewer. The sensor feedback device may include devices such as an acoustic receiving device (e.g., a microphone), a pressure sensor, a joystick, a motion detector, a transducer, and the like. The sensory feedback system may transmit the detected input to controller 520 to coordinate the generation of holographic content and/or sensory feedback.

To illustrate, in an example embodiment of a light field display assembly, the sensory feedback system 570 includes a microphone. The microphone is configured to record audio (e.g., wheezing, screaming, laughing, etc.) produced by one or more viewers. The sensory feedback system 570 provides the recorded audio as viewer input to the controller 520. The controller 520 may generate holographic content using the viewer input. Similarly, sensory feedback system 570 may comprise a pressure sensor. The pressure sensor is configured to measure a force applied to the pressure sensor by a viewer. Sensory feedback system 570 can provide the measured force as a viewer input to controller 520.

In some embodiments, the LF display system 500 includes a tracking system 580. The tracking system 580 includes any number of tracking devices configured to determine the location, movement, and/or characteristics of viewers in the target area. Typically, the tracking device is external to the LF display assembly 510. Some example tracking devices include a camera assembly ("camera"), a depth sensor, a structured light, a LIDAR system, a card scanning system, or any other tracking device that can track a viewer within a target area.

The tracking system 580 may include one or more energy sources that illuminate some or all of the target areas with light. However, in some cases, when rendering holographic content, the target area is illuminated by natural and/or ambient light from LF display assembly 510. The energy source projects light when executing instructions received from the controller 520. The light may be, for example, a structured light pattern, a light pulse (e.g., an IR flash lamp), or some combination thereof. The tracking system may project light of: a visible band (about 380nm to 750nm), an Infrared (IR) band (about 750nm to 1700nm), an ultraviolet band (10nm to 380nm), some other portion of the electromagnetic spectrum, or some combination thereof. The source may comprise, for example, a Light Emitting Diode (LED), a micro LED, a laser diode, a TOF depth sensor, a tunable laser, etc.

The tracking system 580 may adjust one or more transmit parameters when executing instructions received from the controller 520. The emission parameters are parameters that affect how light is projected from the source of the tracking system 580. The emission parameters may include, for example, brightness, pulse rate (including continuous illumination), wavelength, pulse length, some other parameter that affects how light is projected from the source assembly, or some combination thereof. In one embodiment, the source projects a pulse of light in time-of-flight operation.

The camera of tracking system 580 captures an image of the light (e.g., structured light pattern) reflected from the target area. When the tracking instruction received from the controller 520 is executed, the camera captures an image. As previously described, light may be projected by a source of the tracking system 580. The camera may comprise one or more cameras. In other words, the camera may be, for example, an array of photodiodes (1D or 2D), a CCD sensor, a CMOS sensor, some other device that detects some or all of the light projected by the tracking system 580, or some combination thereof. In one embodiment, tracking system 580 may contain a light field camera external to LF display assembly 510. In other embodiments, a camera is included as part of LF display source/sensor module 514 included in LF display assembly 510. For example, as previously described, if the energy relay elements of light field module 512 are bidirectional energy layers that interleave both the emissive display and the imaging sensor at energy device layer 220, then LF display assembly 510 may be configured to simultaneously project the light field and record imaging information from the viewing area in front of the display. In one embodiment, the images captured from the bi-directional energy surface form a light field camera. The camera provides the captured image to the controller 520.

When executing the tracking instructions received from controller 520, the camera of tracking system 580 may adjust one or more imaging parameters. Imaging parameters are parameters that affect how the camera captures an image. The imaging parameters may include, for example, frame rate, aperture, gain, exposure length, frame timing, rolling shutter or global shutter capture mode, some other parameter that affects how the camera captures images, or some combination thereof.

Controller 520 controls LF display assembly 510 and any other components of LF display system 500. The controller 520 includes a data store 522, a network interface 524, a tracking module 526, a viewer profiling module 528, and a light field processing engine 530. In other embodiments, controller 520 includes additional or fewer modules than those described herein. Similarly, functionality may be distributed among modules and/or different entities in a manner different from that described herein. For example, the tracking module 526 may be part of the LF display assembly 510 or the tracking system 580.

Data store 522 is a memory that stores information for LF display system 500. The stored information may include display instructions, tracking instructions, emission parameters, imaging parameters, virtual models of the target region, tracking information, images captured by the camera, one or more viewer profiles, calibration data for the light field display assembly 510, configuration data for the LF display system 510 (including resolution and orientation of the LF module 512), desired viewer geometry, content for graphics creation including 3D models, scenes and environments, textures and textures, and other information that the LF display system 500 may use, or some combination thereof. The data storage 522 is a memory, such as a Read Only Memory (ROM), a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (SRAM), or some combination thereof.

The network interface 524 allows the light field display system to communicate with other systems or environments over a network. In one example, LF display system 500 receives holographic content from a remote light-field display system through network interface 524. In another example, LF display system 500 uses network interface 524 to transmit holographic content to a remote data store.

Tracking module 526 tracks the viewer's viewing of content presented by LF display system 500. To this end, the tracking module 526 generates tracking instructions that control the operation of one or more sources and/or one or more cameras of the tracking system 580 and provides the tracking instructions to the tracking system 580. The tracking system 580 executes the tracking instructions and provides tracking inputs to the tracking module 526.

The tracking module 526 may determine the location of one or more viewers within the target area (e.g., sitting in a chair in a movie theater). The determined position may be relative to, for example, some reference point (e.g., a display surface). In other embodiments, the determined location may be within a virtual model of the target area. The tracked position may be, for example, a tracked position of the viewer and/or a tracked position of a portion of the viewer (e.g., eye position, hand position, etc.). The tracking module 526 uses one or more captured images from the cameras of the tracking system 580 to determine position location. The cameras of tracking system 580 may be distributed around LF display system 500 and may capture stereo images, allowing tracking module 526 to passively track the viewer. In other embodiments, the tracking module 526 actively tracks the viewer. In other words, tracking system 580 illuminates some portion of the target area, images the target area, and tracking module 526 determines the location using time-of-flight and/or structured light depth determination techniques. The tracking module 526 uses the determined position location to generate tracking information.

The tracking module 526 may also receive tracking information as input from a viewer of the LF display system 500. The tracking information may contain body movements corresponding to various input options provided to the viewer by LF display system 500. For example, the tracking module 526 may track the viewer's body movements and assign any of the various movements as input to the LF processing engine 530. Tracking module 526 may provide tracking information to data store 522, LF processing engine 530, viewer profiling module 528, any other component of LF display system 500, or some combination thereof.

To provide context to the tracking module 526, consider an example embodiment of the LF display system 500 displaying a movie in which characters in the movie successfully defeat enemy characters. In response to the scene, the viewer waves a fist in the air to show his excitement. The tracking system 580 may record the movement of the viewer's hand and transmit the record to the tracking module 526. The tracking module 526 tracks the movement of the viewer's hand in the log and sends the input to the LF processing engine 530. As described below, the viewer profiling module 528 determines that the information in the image indicates that the motion of the viewer's hand is associated with a positive response. Thus, if enough viewers are identified with positive responses, LF processing engine 530 generates appropriate holographic content to celebrate hero characters to defeat enemy characters. For example, LF processing engine 530 may project a paper scrap in a scene.

The LF display system 500 includes a viewer profiling module 528 configured to identify and profile viewers. Viewer profiling module 528 generates a profile of a viewer (or viewers) viewing holographic content displayed by LF display system 500. The viewer profiling module 528 generates a viewer profile based in part on the viewer inputs and the monitored viewer behavior, actions, and reactions. The viewer profiling module 528 may access information obtained from the tracking system 580 (e.g., recorded images, videos, sounds, etc.) and process the information to determine various information. In various examples, the viewer profiling module 528 may use any number of machine vision or machine hearing algorithms to determine viewer behavior, actions, and responses. The monitored viewer behavior may include, for example, smiling, cheering, applauding, laughing, frightening, screaming, excitement, backing, other changes in posture, or movement of the viewer, etc.

More generally, the viewer profile may contain any information received and/or determined about the viewer viewing the holographic content from the LF display system. For example, each viewer profile may record the viewer's actions or responses to content displayed by the LF display system 500. Some example information that may be included in a viewer profile is provided below.

In some embodiments, the viewer profile may describe the viewer's response with respect to displayed characters, actors, scenes, and the like. For example, the viewer profile may indicate that the viewer typically has a positive response to a movie showing a handsome starring actor between 25 and 35 years of age.

In some embodiments, the viewer profile may indicate characteristics of the viewer when watching the movie. For example, a viewer in a movie theater is wearing a jersey showing a university sign. In this case, the viewer profile may indicate that the viewer is wearing a jersey and may prefer holographic content associated with the university signs on the jersey. More broadly, viewer characteristics that may be indicated in a viewer profile may include, for example, age, gender, race, clothing, viewing location in the venue, and so forth.

In some embodiments, the viewer profile may indicate the viewer's preferences regarding desired movie and/or theater characteristics. For example, the viewer profile may indicate that the viewer only likes to view holographic content that is appropriate for the age of each of their family. In another example, the viewer profile may indicate a holographic object volume to display holographic content (e.g., on a wall) and indicate that the holographic object volume does not display holographic content (e.g., above its head). The viewer profile may also indicate that the viewer prefers to present a tactile interface in its vicinity, or to avoid the tactile interface.

In another example, the viewer profile indicates a history of movies viewed by a particular viewer. For example, the viewer profiling module 528 determines that a viewer or group of viewers has previously viewed a movie. Thus, the LF display system 500 can display different holographic content than when the viewer previously viewed the movie. As an example, a movie containing holographic content may have three different endings, and LF display system 500 may display the different endings based on the present viewers. In another example, each of the three endings may be presented to different viewers in the same theater.

In some embodiments, a viewer profile may also describe characteristics and preferences of a group of viewers rather than a particular viewer. For example, the viewer profiling module 528 may generate a viewer profile for a viewer watching a movie in a movie theater. In one example, the viewer profiling module 528 creates a viewer profile for viewers watching movies of teenagers lodickins. The profile indicates that 86.3% of the viewers are women between the ages of 20 and 35 and have a positive response to the movie. The profile also indicates that the remaining 23.7% of viewers are males between the ages of 20 and 35 and have a negative response to the movie. Any of the information and characteristics described previously may be applied to a group of viewers.

The viewer profiling module 528 may also access profiles associated with a particular viewer (or viewers) from one or more third party systems to establish a viewer profile. For example, a viewer purchases a ticket for a movie using a third party provider linked to the viewer's social media account. Thus, the viewer's ticket is linked to his social media account. When a viewer enters a performance venue using their ticket, the viewer profiling module 528 may access information from their social media account to build (or enhance) a viewer profile.

In some embodiments, the data store 522 includes a viewer profile store that stores viewer profiles generated, updated, and/or maintained by the viewer profiling module 528. The viewer profile may be updated in the data store at any time by the viewer profiling module 528. For example, in one embodiment, when a particular viewer views holographic content provided by LF display system 500, viewer profile storage receives and stores information about the particular viewer in its viewer profile. In this example, the viewer profiling module 528 contains a facial recognition algorithm that can identify a viewer and positively identify the viewer as he views the presented holographic content. To illustrate, the tracking system 580 obtains an image of the viewer as the viewer enters the target area of the LF display system 500. The viewer profiling module 528 inputs the captured image and uses a facial recognition algorithm to identify the viewer's face. The identified face is associated with a viewer profile in a profile store, and as such, all of the obtained input information about the viewer may be stored in its profile. The viewer profiling module may also positively identify the viewer using a card identification scanner, a voice identifier, a Radio Frequency Identification (RFID) chip scanner, a bar code scanner, or the like.

In embodiments where the viewer profiling module 528 can positively identify the viewer, the viewer profiling module 528 can determine each viewer's visit to the LF display system 500. The viewer profiling module 528 may then store the time and date of each visit in the viewer profile of each viewer. Similarly, the viewer profiling module 528 may store input received from the viewer at each of its occurrences from any combination of the sensory feedback system 570, the tracking system 580, and/or the LF display assembly 510. The viewer profile system 528 may additionally receive other information about the viewer from other modules or components of the controller 520, which may then be stored with the viewer profile. Other components of controller 520 may then also access the stored viewer profile to determine subsequent content to provide to the viewer.

LF processing engine 530 generates holographic content that includes light field data as well as data for all the sensory domains supported by LF display system 500. For example, LF processing engine 530 may generate 4D coordinates in a rasterized format ("rasterized data"), which, when executed by LF display assembly 510, cause LF display assembly 510 to render holographic content. LF processing engine 530 may access rasterized data from data store 522. Additionally, the LF processing engine 530 may construct rasterized data from the vectorized data set. Vectorized data is described below. LF processing engine 530 may also generate the sensory instructions needed to provide the sensory content that enhances the holographic objects. As described above, when executed by LF display system 500, the sensory instructions may generate tactile surfaces, sound fields, and other forms of sensory energy supported by LF display system 500. The LF processing engine 530 may access sensory instructions from the data store 522, building sensory instructions from the vectorized data set. The 4D coordinates and sensory data collectively represent the holographic content as display instructions executable by the LF display system to generate holographic and sensory content. More generally, the holographic content may take the form of CG content having: ideal light field coordinates, live action content, rasterized data, vectorized data, electromagnetic energy transmitted by a set of relays, instructions sent to a set of energy devices, energy locations on one or more energy surfaces, a set of energy propagation paths projected from a display surface, holographic objects visible to a viewer or audience, and many other similar forms.

The amount of rasterized data describing the flow of energy through the various energy sources in the LF display system 500 is very large. Although rasterized data may be displayed on the LF display system 500 when accessed from the data store 522, rasterized data may not be efficiently transmitted, received (e.g., via the network interface 524), and subsequently displayed on the LF display system 500. For example, take the example of rasterized data representing a short slice of holographic projection by LF display system 500. In this example, the LF display system 500 includes a display that contains billions of pixels, and the rasterized data contains information for each pixel location of the display. The corresponding size of the rasterized data is huge (e.g., several gigabytes per second of movie display time) and is not manageable for efficient delivery over the business network through the network interface 524. For real-time streaming applications involving holographic content, the problem of efficient delivery can be magnified. When an interactive experience is required using input from the sensory feedback system 570 or the tracking module 526, an additional problem arises of storing only rasterized data on the data storage device 522. To enable an interactive experience, the light field content generated by LF processing engine 530 may be modified in real-time in response to sensory or tracking inputs. In other words, in some cases, the LF content cannot simply be read from the data storage 522.

Thus, in some configurations, data representing holographic content displayed by LF display system 500 may be passed to LF processing engine 530 in a vectorized data format ("vectorized data"). Vectorized data may be orders of magnitude smaller than rasterized data. Furthermore, vectorized data provides high image quality while having a data set size that enables efficient sharing of data. For example, the vectorized data may be a sparse data set derived from a denser data set. Thus, based on how sparse vectorized data is sampled from dense rasterized data, the vectorized data may have an adjustable balance between image quality and data transfer size. The adjustable sampling for generating vectorized data enables optimization of image quality at a given network speed. Thus, efficient transmission of holographic content over the network interface 524 is achieved for the vectorized data. The vectorized data also enables real-time streaming of the holographic content over commercial networks.

In summary, the LF processing engine 530 may generate holographic content derived from rasterized data accessed from the data storage 522, vectorized data accessed from the data storage 522, or vectorized data received through the network interface 524. In various configurations, the vectorized data may be encoded prior to data transmission and decoded after reception by the LF controller 520. In some examples, the quantized data is encoded for additional data security and performance improvements related to data compression. For example, the vectorized data received over the network interface may be encoded vectorized data received from a holographic streaming application. In some instances, the vectorized data may require a decoder, the LF processing engine 530, or both to access the information content encoded in the vectorized data. The encoder and/or decoder system may be available for use by a consumer or authorized for a third party vendor.

The vectorized data contains all the information for each sensory domain that the LF display system 500 supports in a way that can support an interactive experience. For example, vectorized data for an interactive holographic experience may include any vectorized feature that may provide an accurate physical effect for each sensory domain supported by LF display system 500. Vectorized features may include any feature that may be synthetically programmed, captured, evaluated computationally, and the like. The LF processing engine 530 may be configured to convert vectorized features in the vectorized data into rasterized data. LF processing engine 530 may then project the holographic content converted from the vectorized data using LF display assembly 510. In various configurations, vectorized features may include: one or more red/green/blue/alpha channels (RGBA) + depth images; a plurality of view images of depth information with or without different resolutions, which view images may contain one high resolution center image and other views of lower resolution; material characteristics such as albedo and reflectance; a surface normal; other optical effects; surface identification; geometric object coordinates; virtual camera coordinates; displaying the plane position; an illumination coordinate; the tactile stiffness of the surface; tactile malleability; the strength of the touch; amplitude and coordinates of the sound field; (ii) an environmental condition; somatosensory energy vectors associated with mechanical receptors for texture or temperature, audio; as well as any other sensory domain characteristics. Many other vectorized features are possible.

The LF display system 500 may also generate an interactive viewing experience. That is, the holographic content may be responsive to input stimuli containing information about viewer location, gestures, interactions with the holographic content, or other information originating from viewer profiling module 528 and/or tracking module 526. For example, in an embodiment, LF processing system 500 creates an interactive viewing experience using real-time performance managed vectorized data received through network interface 524. In another example, if a holographic object needs to be moved in a particular direction immediately in response to a viewer interaction, LF processing engine 530 may update the rendering of the scene so the holographic object moves in the desired direction. This may require the LF processing engine 530 to use the vectorized data set to render the light field in real-time based on the 3D graphics scene with appropriate object placement and movement, collision detection, occlusion, color, shading, lighting, etc., to correctly respond to viewer interactions. LF processing engine 530 converts the vectorized data into rasterized data for rendering by LF display assembly 510.

The rasterized data contains holographic content instructions and sensory instructions (display instructions) that represent real-time performance. LF display assembly 510 simultaneously projects real-time performance holographic and sensory content by executing display instructions. The LF display system 500 monitors viewer interactions (e.g., voice responses, touches, etc.) with the presented real-time performance through the tracking module 526 and the viewer profiling module 528. In response to the viewer interaction, the LF processing engine may create an interactive experience by generating additional holographic and/or sensory content for display to the viewer.

To illustrate, consider an example embodiment of an LF display system 500 that includes an LF processing engine 530 that generates a plurality of holographic objects representing balloons that have dropped from the ceiling of a movie theater during a movie. The viewer may move to touch the holographic object representing the balloon. Accordingly, tracking system 580 tracks the movement of the viewer's hand relative to the holographic object. The movement of the viewer is recorded by the tracking system 580 and sent to the controller 520. The tracking module 526 continuously determines the movement of the viewer's hand and sends the determined movement to the LF processing engine 530. The LF processing engine 530 determines the placement of the viewer's hand in the scene, adjusting the real-time rendering of the graphics to include any desired changes (such as positioning, color, or occlusion) in the holographic object. LF processing engine 530 instructs LF display assembly 510 (and/or sensory feedback system 570) to generate a tactile surface using a volumetric tactile projection system (e.g., using an ultrasonic speaker). The generated tactile surface corresponds to at least a portion of the holographic object and occupies substantially the same space as some or all of the external surfaces of the holographic object. The LF processing engine 530 uses the tracking information to dynamically instruct the LF display assembly 510 to move the location of the haptic surface along with the location of the rendered holographic object so that the viewer is given both visual and tactile perceptions of touching the balloon. More simply, when a viewer sees his hand touching the holographic balloon, the viewer simultaneously feels tactile feedback indicating that his hand touched the holographic balloon, and the balloon changes position or motion in response to the touch. In some examples, rather than presenting an interactive balloon in a movie accessed from data storage 522, the interactive balloon may be received as part of holographic content received from a real-time streaming application through network interface 524.

LF processing engine 530 may provide holographic content for simultaneous display to viewers in a movie theater watching a movie. For example, a movie shown in a movie theater enhanced with LF display system 500 contains holographic content ("holographic content tracks") to be presented to viewers during the movie. The holographic content track may be received by a publisher of the movie and stored in data storage 522. The holographic content track contains holographic content that enhances the viewing experience of viewers watching a movie in a movie theater.

The holographic content in the holographic content track may be associated with any number of temporal, auditory, visual, etc. cues to display the holographic content. For example, a holographic content track may contain holographic content to be displayed at a particular time during a movie. By way of illustration, a holographic content track comprises a group of holographic dolphins displayed 35 minutes 42 seconds after the start of the movie during the movie "bottomised". In another example, the holographic content track includes holographic content that will be presented when the sensory feedback system 570 records a particular audio prompt. By way of illustration, the holographic content track includes a holographic ghosted turkey that would appear when the audio recorded by the sensory feedback assembly 570 during the movie "Poultrygeist" indicates that a person is laughing during the movie. In another example, the holographic content track includes holographic content that will be displayed when tracking system 580 records a particular visual cue. By way of illustration, the holographic content track is contained in the movie "Don't Leave Me, Bro! "during which the" away person!will be displayed when the tracking system 580 records information indicating that the viewer is away from the movie theater! "holographic mark. Determining the audible and visual cues is described in more detail below.

The holographic content track may also contain spatial rendering information. That is, the holographic content track may indicate a spatial location for rendering the holographic content in a movie theater. For example, a holographic content track may indicate that some holographic content is to be presented in some holographic viewing volumes, but not in others. To illustrate, LF processing engine 530 may present the death of crow in a holographic viewer on the viewer's head rather than on the movie theater wall. Similarly, a holographic content track may indicate that holographic content is presented to some viewing volumes, but not other viewing volumes. For example, the LF processing engine may present the death of crow's eyes to viewers near the back of the theater and butterfly's kaleidoscope to viewers near the front of the theater.

LF processing engine 530 may provide holographic content for display to viewers in a movie theater before, during, and/or after the movie to enhance the movie theater experience. The holographic content may be provided by the publisher of the movie, by the movie theater, by the advertiser, generated by the LF processing engine 530, etc. The holographic content may be content associated with a movie, a genre of the movie, a location of the movie, advertisements, etc. In any case, the holographic content may be stored in data storage 522 or streamed to LF display system 500 in a vectorized format over network interface 524. For example, a movie may be projected on a wall in a movie theater enhanced with an LF display module. The publisher of the movie may provide the holographic content presented on the wall display before the movie starts. The LF processing engine 530 accesses the holographic content and renders the accessed content from the display onto the wall of the movie theater prior to the start of the movie. In another example, a movie theater with LF display system 500 is located in san francisco. If the movie specific content is not provided, the movie theater stores a holographic representation of the gold gate bridge that was presented in the movie theater prior to the movie. Here, since no movie-specific holographic content is provided, LF processing engine 530 accesses the golden gate bridge and renders the golden gate bridge in the movie theater. In another example, advertisers have provided holographic content of their products as advertisements to movie theaters for display after a movie. After the movie ends, the LF processing engine 530 presents the advertisement to the viewer as he leaves the movie theater. In other examples, the LF processing engine may dynamically generate holographic content for display on a wall of a theater, as described below.

LF processing engine 500 may also modify the holographic content to suit the theater in which the holographic content is being presented. For example, not every theater may have the same size, the same number of seats, or the same technical configuration. Thus, LF processing engine 530 may modify the holographic content so that it will be properly displayed in the theater. In one embodiment, the LF processing engine 530 may access a configuration file for the theater, including the theater's layout, resolution, field of view, other specifications, and the like. LF processing engine 530 may render and present holographic content based on information contained in the configuration file.

LF processing engine 530 may also create holographic content for display by LF display system 500. Importantly, here, creating holographic content for display is different from accessing or receiving holographic content for display. In other words, when creating content, the LF processing engine 530 generates entirely new content for display, rather than accessing previously generated and/or received content. LF processing engine 530 may use information from tracking system 580, sensory feedback system 570, viewer profiling module 528, tracking module 526, or some combination thereof, to create holographic content for display. In some instances, LF processing engine 530 may access information from elements of LF display system 500 (e.g., tracking information and/or viewer profiles), create holographic content based on the information, and in response, display the created holographic content using LF display system 500. The created holographic content may be enhanced with other sensory content (e.g., touch, audio, or scent) when displayed by LF display system 500. Further, the LF display system 500 may store the created holographic content so that it may be displayed in the future.

Dynamic content generation for LF display systems

In some embodiments, LF processing engine 530 incorporates Artificial Intelligence (AI) models to create holographic content for display by LF display system 500. The AI model may include supervised or unsupervised learning algorithms, including but not limited to regression models, neural networks, classifiers, or any other AI algorithm. The AI model may be used to determine viewer preferences based on viewer information recorded by the LF display system 500 (e.g., by the tracking system 580), which may contain information about the viewer's behavior.

The AI model may access information from data store 522 to create holographic content. For example, the AI model may access viewer information from a viewer profile or profiles in data store 522 or may receive viewer information from various components of LF display system 500. To illustrate, the AI model may determine that the viewer likes to view holographic content in which the actor wears a bow tie. The AI model may determine preferences based on positive reactions or responses of a group of viewers to previously viewed holographic content including an actor wearing a bow tie. In other words, the AI model may create holographic content personalized for a group of viewers according to learned preferences of those viewers. Thus, for example, the AI model may incorporate a bow tie into an actor displayed in holographic content viewed by a group of viewers using LF display system 500. The AI model may also store learned preferences for each viewer in a viewer profile store of data store 522. In some instances, the AI model may create holographic content for a single viewer rather than a group of viewers.

One example of an AI model that may be used to identify characteristics of a viewer, identify responses, and/or generate holographic content based on the identified information is a convolutional neural network model having a layer of nodes, where the values at the nodes of a current layer are transforms of the values at the nodes of a previous layer. The transformation in the model is determined by a set of weights and parameters that connect the current layer and the previous layer. For example, and the AI model may contain five levels of nodes: layers A, B, C, D and E. The transformation from layer A to layer B is represented by the function W₁Given that the transformation from layer B to layer C is represented by W₂Given that the transformation from layer C to layer D is represented by W₃Give aAnd the conversion from layer D to layer E is represented by W₄It is given. In some instances, the transformation may also be determined by a set of weights and parameters used to transform between previous layers in the model. For example, the transformation W from layer D to layer E₄May be based on the transformation W used to complete the layer A to B₁The parameter (c) of (c).

The input to the model may be the image encoded onto convolutional layer a acquired by tracking system 580, and the output of the model is the holographic content decoded from output layer E. Alternatively or additionally, the output may be a determined characteristic of the viewer in the image. In this example, the AI model identifies potential information in the image that represents the viewer characteristics in identification layer C. The AI model reduces the dimensionality of convolutional layer a to the dimensionality of identification layer C to identify any features, actions, responses, etc. in the image. In some instances, the AI model then adds the dimensions of identification layer C to generate the holographic content.

The image from the tracking system 580 is encoded into convolutional layer a. The image input in convolutional layer a may be related to various characteristics and/or reaction information, etc. in identification layer C. The relevant information between these elements can be retrieved by applying a set of transformations between the corresponding layers. That is, the convolution layer a of the AI model represents the encoded image, and the identification layer C of the model represents the smiling viewer. Can be obtained by transforming W₁And W₂Applied to the pixel values of the images in the space of convolutional layer a to identify a smiling viewer in a given image. The weights and parameters used for the transformation may indicate the relationship between the information contained in the image and the identity of the smiling viewer. For example, the weights and parameters may be quantifications of shapes, colors, sizes, etc. contained in information representing a smiling viewer in an image. The weights and parameters may be based on historical data (e.g., previously tracked viewers).

The smiley viewer in the image is identified in the identification layer C. Identification layer C represents a smiling viewer identified based on potential information about the smiling viewer in an image

The identified smiley viewer in the image may be used to generate holographic content. To generate holographic content, the AI model is at the identification levelStart at C and transform W₂And W₃The value applied to identify a given identified smiling viewer in layer C. The transformation produces a set of nodes in the output layer E. The weights and parameters for the transformation may indicate a relationship between the identified smiling viewer and the particular holographic content and/or preferences. In some cases, the holographic content is output directly from the node of output layer E, while in other cases, the content generation system decodes the node of output layer E into the holographic content. For example, if the output is a set of identified characteristics, the LF processing engine may use the characteristics to generate holographic content.

In addition, the AI model may contain layers referred to as intermediate layers. An intermediate layer is a layer that does not correspond to an image, does not identify a property/reaction, etc., or does not generate holographic content. For example, in the given example, layer B is an intermediate layer between convolutional layer a and identification layer C. Layer D is an intermediate layer between identification layer C and output layer E. The hidden layer is a potential representation of different aspects of the identification that are not observable in the data, but can control the relationship between the image elements when identifying the characteristics and generating the holographic content. For example, a node in the hidden layer may have a strong connection (e.g., a large weight value) with the input value and the identification value that share the commonality of "happy man smiling". As another example, another node in the hidden layer may have a strong connection with the input value and the identification value that share a commonality of "scary screaming". Of course, there are any number of connections in the neural network. In addition, each intermediate layer is a combination of functions, such as a residual block, convolutional layer, pooling operation, skip connection, series, and the like. Any number of intermediate layers B may be used to reduce the convolutional layers to the identification layers, and any number of intermediate layers D may be used to add the identification layers to the output layers.

In one embodiment, the AI model contains a deterministic method that has been trained with reinforcement learning (thereby creating a reinforcement learning model). The model is trained to use measurements from tracking system 580 as input and changes in the created holographic content as output to improve the quality of performance.

Reinforcement learning is a machine learning system in which the machine learns "what to do" -how to map cases to actions-in order to maximize the digital reward signal. Rather than informing the learner (e.g., LF processing engine 530) which actions to take (e.g., generating specified holographic content), attempts to find out which actions yield the highest rewards (e.g., by cheering more people to improve the quality of the holographic content) are performed. In some cases, the action may affect not only the instant reward, but also the next case, and therefore all subsequent rewards. These two features-trial false searches and delayed rewards-are two significant features of reinforcement learning.

Reinforcement learning is defined not by characterizing the learning method, but by characterizing the learning problem. Basically, reinforcement learning systems capture those important aspects of the problem facing learning agents interacting with their environment to achieve goals. That is, in the example of generating a song for a performer, the reinforcement learning system captures information about viewers in the venue (e.g., age, personality, etc.). The agent senses the state of the environment and takes action that affects the state to achieve one or more goals (e.g., creating a popular song that the viewer cheers). In its most basic form, the formulation of reinforcement learning encompasses three aspects of the learner: sensation, action, and goal. Continuing with the song example, LF processing engine 530 senses the state of the environment through sensors of tracking system 580, displays holographic content to viewers in the environment, and achieves a goal that is a measure of the reception of the song by the viewers.

One of the challenges that arises in reinforcement learning is the tradeoff between exploration and utilization. To increase rewards in the system, reinforcement learning agents prefer actions that have been tried in the past and found to be effective in generating rewards. However, to discover the actions that generate the reward, the learning agent may select actions that were not previously selected. The agent "leverages" the information it already knows to obtain rewards, but it also "explores" the information to make better action choices in the future. The learning agent attempts various actions and gradually favors those actions that look best while continuing to attempt new actions. On a random task, each action is typically tried multiple times to obtain a reliable estimate of its expected reward. For example, if the LF processing engine creates holographic content that the LF processing engine knows will cause the viewer to laugh after a long period of time, the LF processing engine may change the holographic content such that the time until the viewer laughs is reduced.

In addition, reinforcement learning considers the entire problem of target-oriented agent interaction with uncertain environments. Reinforcement learning agents have specific goals that can sense aspects of their environment and can choose to receive high reward actions (i.e., growers). Furthermore, agents are typically still operating despite significant uncertainty in the environment they are confronted with. When reinforcement learning involves planning, the system will address the interaction between the planning and the real-time action selection, and how to acquire and improve the environmental elements. In order to advance reinforcement learning, important sub-problems must be isolated and studied, which play a clear role in the complete interactive target-seeking agent.

Reinforcement learning problems are a framework of machine learning problems in which interactions are processed and actions are performed to achieve a goal. Learners and decision makers are referred to as agents (e.g., LF processing engine 530). Things that an agent interacts with (including everything beyond the agent) are referred to as environments (e.g., viewers in a venue, etc.). The two continuously interact, the agent selects actions (e.g., creating holographic content), and the environment responds to these actions and presents the new situation to the agent. The environment also brings rewards, i.e., special values that the agent tries to maximize over time. In one context, rewards serve to maximize the viewer's positive response to the holographic content. The complete specification of the environment defines a task that is an example of a reinforcement learning problem.

To provide more context, the agent (e.g., LF processing engine 530) interacts with the environment in each discrete time step in a series of discrete time steps (i.e., t ═ 0, 1, 2, 3, etc.). At each time step t, the agent receives some environmental state s_tFor example, measurements from the tracking system 580. State s_tWithin S, where S is a set of possible states. Based on the state s_tAnd a time step t, the agent selects an action at (e.g., having the actor split). Action at is in A(s)_t) Wherein A(s)_t) Is a set of possible actions. At a later time state (in part as a result of its action), the agent receives a digital award r_t+1. State r_t+1Within R, where R is the set of possible rewards. Once the agent receives the reward, the agent selects a new state s_t+1。

At each time step, the agent implements a mapping from states to probabilities of selecting each possible action. This mapping is called a proxy policy and is denoted as π_tIn which pi_t(s, a) is if s_tA when is ═ s_tProbability of a. The reinforcement learning method may decide how an agent changes its policy due to the state and rewards generated by the agent's actions. The objective of the agent is to maximize the total number of rewards received over time.

This reinforcement learning framework is very flexible and can be applied to many different problems in many different ways (e.g., generating holographic content). The framework suggests that whatever the details of the sensory, memory and control devices, any problem (or purpose) of learning a target-oriented behavior can be reduced to three signals that are passed back and forth between the agent and its environment: one signal indicates the selection (action) made by the agent, one signal indicates the basis on which the selection was made (status), and one signal defines the agent goal (reward).

Of course, the AI model may include any number of machine learning algorithms. Some other AI models that may be employed are linear and/or logistic regression, classification and regression trees, k-means clustering, vector quantization, and the like. Regardless, the LF processing engine 530 typically takes input from the tracking module 526 and/or the viewer profiling module 528 and in response the machine learning model creates holographic content. Similarly, the AI model may direct the presentation of holographic content.

LF processing engine 530 may create holographic content based on a movie being shown in a movie theater. For example, a movie shown in a movie theater may be associated with a set of metadata that describes the characteristics of the movie. The metadata may include, for example, environment, genre, actors, theme, title, run time, rating, and the like. LF processing engine 530 may access any of the metadata describing the movie and, in response, generate holographic content for presentation in the movie theater. For example, a movie entitled "The Last Merman" would be played in a movie theater enhanced with LF display system 500. The LF processing engine 530 accesses the metadata of the movie to create holographic content for the wall of the movie theater prior to the start of the movie. Here, the metadata containing environment is underwater and the type is romantic. LF processing engine 530 enters the metadata into the AI model and in response receives the holographic content displayed on the wall of the movie theater. In this example, LF processing engine 530 creates a seaside sunset that is displayed on the wall of the movie theater before the movie begins playing.

In an example, LF processing engine 530 may convert a traditional two-dimensional (2D) movie into holographic content for display by an LF display system. For example, LF processing engine 530 may input a traditional movie into the AI model, and the AI model converts any portion of the traditional movie into holographic content. In an example, the AI model may convert a traditional movie into holographic content by using a machine learning algorithm trained by converting two-dimensional data into holographic data. In various scenarios, the training data may be previously generated, created, or some combination of the two. The LF display system 500 may then display a holographic version of the movie instead of the conventional two-dimensional version of the movie.

In an example, LF processing engine 530 creates holographic content based on viewers present at a movie theater containing LF display system 500. For example, a group of viewers enters a movie theater to view a movie enhanced with holographic content displayed by LF display system 500. The viewer profiling module 528 generates viewer profiles for viewers in the theater and an aggregate viewer profile representing all viewers in the theater. The LF processing engine 530 accesses the aggregated viewer profile and creates holographic content for display to viewers in the movie theater. For example, the viewers in a movie theater are a group of couples who watch romantic movies, and thus, the aggregate viewer profile contains information (e.g., by parameterization and input to the AI model) indicating that they may like holographic content that is commensurate with couples in the appointment. Thus, the LF processing engine 530 generates holographic content such that the movie theater has a more romantic atmosphere (e.g., candles, dim lights, Marvin Gaye's music, etc.).

In an example, LF processing engine 530 creates holographic content based on responses of viewers watching the movie. For example, a viewer in a movie theater is watching a movie in a movie theater augmented by LF display system 500. The tracking module 526 and the viewer profiling module 528 monitor the responses of viewers watching the movie. For example, the tracking system 580 may obtain an image of a viewer while watching a movie. The tracking module 526 identifies the viewer, and the viewer profiling module 528 uses machine vision algorithms to determine the viewer's response based on the information contained in the image. For example, the AI model may be used to identify whether a viewer watching a movie is smiling, and thus, the viewer profiling module 528 may indicate in the viewer profile whether the viewer responds positively or negatively to the movie based on the smiling. Other reactions can also be determined. By way of illustration, a movie theater is showing a comedy movie. The tracking module 526 and the viewer profiling module 528 monitor the viewer's responses as the movie is shown on the display surface. In this case, the viewer is learning the montage of playing piano from a cat in a jeer movie. The viewer profiling module 528 identifies laughter and in response the LF processing engine 530 displays holographic content of kitten doing fool on the wall of the theater.

In a similar example, LF processing engine 530 may create holographic content based on pre-existing or provided advertising content. In other words, for example, LF processing engine 530 may request an advertisement from a network system through network interface 524, in response, the network system provides holographic content, and LF processing engine 530 creates holographic content for display containing the advertisement. Some examples of advertisements may include products, text, video, and the like. Advertisements may be presented to particular viewers based on the viewers in the viewers. Similarly, holographic content may enhance a movie with advertisements (e.g., product placement). Most generally, as previously described, the LF processing engine 530 may create advertising content based on any of the characteristics and/or reactions of the viewers in the scene.

The foregoing examples of creating content are not limiting. Most broadly, LF processing engine 530 creates holographic content for display to a viewer of LF display system 500. Holographic content may be created based on any of the information contained in LF display system 500.

Holographic content distribution network

Fig. 5B shows an example LF movie network 550 in accordance with one or more embodiments. One or more LF display systems may be included in the LF movie network 550. The LF movie network 550 contains any number of LF display systems (e.g., 500A, 500B, and 500C), LF movie generation systems 554, and networking systems 556 coupled to one another by a network 552. In other embodiments, LF movie network 550 includes additional or fewer entities than those described herein. Similarly, functionality may be distributed among different entities in a different manner than described herein.

In the illustrated embodiment, the LF movie network 550 includes LF display systems 500A, 500B, and 500C that can receive holographic content over the network 552 and display the holographic content to viewers. LF display systems 500A, 500B, and 500C are collectively referred to as LF display system 500.

LF movie generation system 554 is a system that generates holographic content for display in a movie theater containing an LF display system. The holographic content may be a movie or may be holographic content that enhances a traditional movie. LF movie generation system 554 may include any number of sensors and/or processors to record information and generate holographic content. For example, the sensors may include a camera for recording images, a microphone for recording audio, a pressure sensor for recording interactions with objects, and so forth. The LF movie generation system 554 may also be a computer system that generates synthesized light field data using a processor to generate a computer-generated imaging (CGI) animation that is rendered as a light field. The synthetic light field data may be from, for example, a virtual world or an animated movie. LF movie generation system 554 combines the recorded information and encodes the information into holographic and perceptual content. LF movie generation system 554 may transmit the encoded holographic content to one or more of LF display systems 500 for display to a viewer. As previously discussed, to achieve an effective transfer speed, the data of the LF display systems 500A, 500B, 500C, etc. may be transferred as vectorized data over the network 552.

More broadly, LF movie generation system 554 generates holographic content for display in a movie theater by using any recorded sensory data or synthetic data that may be used by the LF display system in showing the movie. For example, sensory data may include recorded audio, recorded images, recorded interactions with objects, and so forth. Many other types of sensory data may be used. To illustrate, the recorded visual content may include: 3D graphical scenes, 3D models, object placement, texture, color, shading, and lighting; large datasets that can be converted using AI models and similar cinematic transformations are converted into 2D cinematic data in holographic form; multi-view camera data from a camera rig having a number of cameras with or without depth channels; plenoptic camera data; a CG content; or many other types of content.

In some configurations, the LF movie generation system 554 may use a proprietary encoder to perform encoding operations that reduce the sensory data recorded for the movie to a vectorized data format as described above. That is, encoding the data as vectorized data may include image processing, audio processing, or any other computation that may result in a reduced data set that is easier to transmit over the network 552. The encoder may support a format used by professionals in the motion picture production industry.

Each LF display system (e.g., 500A, 500B, 500C) may receive encoded data from the network 552 through the network interface 524. In this example, each LF display system includes a decoder to decode the encoded LF display data. More specifically, LF processing engine 530 generates rasterized data for LF display assembly 510 by applying a decoding algorithm provided by a decoder to the received encoded data. In some examples, the LF processing engine may additionally generate rasterized data for the LF display assembly using input from the tracking module 526, the viewer profiling module 528, and the sensory feedback system 570 as described herein. The holographic content recorded by LF movie generation system 554 is reproduced for rasterized data generated by LF display assembly 510. Importantly, each LF display system 500A, 500B, and 500C generates rasterized data appropriate for the particular configuration of the LF display assembly in terms of geometry, resolution, and the like. In some configurations, the encoding and decoding processes are part of a proprietary encoding/decoding system pair that may be provided to a display consumer or authorized by a third party. In some cases, the encoding/decoding system pair may be implemented as a proprietary API that may provide a common programming interface to content creators.

In some configurations, the various systems in the LF movie network 550 (e.g., the LF display system 500, the LF movie generation system 554, etc.) may have different hardware configurations. The hardware configuration may include an arrangement of physical systems, energy sources, energy sensors, haptic interfaces, sensory capabilities, resolution, LF display module configuration, or any other hardware description of the systems in the LF cinema network 550. Each hardware configuration may generate or utilize sensory data in a different data format. Thus, the decoder system may be configured to decode encoded data for the LF display system to be presented on the LF display system. For example, an LF display system having a first hardware configuration (e.g., LF display system 500A) receives encoded data from an LF movie generation system having a second hardware configuration (e.g., LF movie generation system 554). The decoding system accesses information describing the first hardware configuration of the LF display system 500A. The decoding system decodes the encoded data using the accessed hardware configuration so that the decoded data can be processed by the LF processing engine 530 of the receiving LF display system 500A. While recorded in the second hardware configuration, LF processing engine 530 generates and renders rasterized content for the first hardware configuration. In a similar manner, regardless of the hardware configuration, holographic content recorded by LF movie generation system 554 may be rendered by any LF display system (e.g., LF display system 500B, LF display system 500C).

Network system 556 is any system configured to manage holographic content transmission between systems in LF movie network 550. For example, network system 556 may receive a request for holographic content from LF display system 500A and facilitate the transmission of holographic content from LF movie generation system 554 to LF display system 500A. The network system 556 may also store holographic content, viewer profiles, holographic content, etc. for transmission to and/or storage by other LF display systems 500 in the LF movie network 550. The network system 556 may also contain an LF processing engine 530 that can create holographic content as previously described.

The network system 556 may include a Digital Rights Management (DRM) module to manage the digital rights of the holographic content. For example, LF movie generation system 554 may transmit the holographic content to network system 556, and the DRM module may encrypt the holographic content using a digital encryption format. In other examples, LF movie generation system 554 encodes the recorded light field data into a holographic content format that can be managed by a DRM module. Network system 556 may provide digitally encrypted keys to LF display systems so that each LF display system 500 can decrypt and subsequently display the holographic content to a viewer. Most generally, the network system 556 and/or the LF movie generation system 554 encode the holographic content, and the LF display system can decode the holographic content.

Network system 556 may act as a repository for previously recorded and/or created holographic content. Each piece of holographic content may be associated with a transaction fee that, when received, causes network system 556 to transmit the holographic content to LF display system 500 that provides the transaction fee. For example, LF display system 500A may request access to holographic content over network 552. The request includes a transaction fee for the holographic content. In response, network system 556 transmits the holographic content to the LF display system for display to the viewer. In other examples, the network system 556 may also serve as a subscription service for holographic content stored in the network system. In another example, LF recording system 554 records light field data of a performance in real-time and generates holographic content representing the performance. LF display system 500 transmits a request for holographic content to LF recording system 554. The request includes a transaction fee for the holographic content. In response, LF recording system 554 transmits the holographic content for simultaneous display on LF display system 500. Network system 556 may act as a mediator in exchanging transaction fees and/or managing the holographic content data stream across network 552.

Network 552 represents the communication paths between systems in LF movie network 550. In one embodiment, the network is the internet, but may be any network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wired, or wireless network, a cloud computing network, a private network, or a virtual private network, and any combination thereof. Additionally, all or some of the links may be encrypted using conventional encryption techniques, such as Secure Sockets Layer (SSL), encrypted HTTP, and/or Virtual Private Networks (VPNs). In another embodiment, the entities may use custom and/or dedicated data communication techniques instead of or in addition to those described above.

Movie theatre including LF display system

Fig. 6-8 show several example movie theaters that may display holographic content using an LF display system (e.g., LF display system 500). As previously described, the holographic content may include a movie and/or any other holographic content that may enhance the experience of watching a movie in a movie theater. Within a movie theater, any number of viewers are located at viewing positions within any number of viewers. The LF display system is configured to display holographic content in a holographic object volume within the movie theater such that a viewer in the viewing volume can perceive the holographic content. Typically, an LF display system in a movie theater includes one or more LF display assemblies (e.g., LF display assembly 510), each containing an array of LF display modules (e.g., LF display module 512) forming a seamless polymeric surface. This polymeric surface may be located on multiple surfaces having various relative orientations, for example, on two orthogonal walls. Thus, the holographic object in the movie theater may be an aggregated holographic object display of an array of LF display modules.

Fig. 6A shows a theater 600A including an LF display system in accordance with one or more embodiments. Some or all of the walls of theater 600A are lined with an array of LF display modules. The array of LF display modules forms the polymeric surface of LF display 620. For example, as shown, front wall 626 of the theater contains LF display 620. Although not shown, in some examples, the side walls of the theater may also include portions of the LF display. Theater 600A includes several viewing locations 642 arranged in rows and positioned to view holographic content presented by LF display 620. Viewing position 642 is the portion of viewing volume 650 from which holographic objects rendered by the LF display system (not occluded) are fully viewable. For example, a user at a particular viewing location, such as viewing location 642A, will be able to view holographic objects presented in the holographic object volume of LF display 620. In the illustrated embodiment, at least some of the viewing positions 642 include a chair (e.g., viewing position 642A) in which a viewer sits. There are also some positions without chairs. Viewing location 642 in theater 600A is also positioned such that a viewer in viewing location 642 may perceive holographic content presented from LF display 620. Although not shown, some or all of the ceiling, floor, or both may also be lined with an LF display 620. In fig. 6A, the LF display system is an embodiment of the LF display system 500. In addition, the LF display module in the LF display 620 is an embodiment of the LF display module 512, and may be the same as the LF display module 300A, 210, or 110.

The LF display system in theater 600A is optimized for a particular viewing volume (e.g., viewing volume 650). In the example shown, the viewing volume 650 contains all of the viewing positions 642 such that a viewer at a viewing position can view holographic content presented by LF display 620. As shown, LF display 620 is mounted on a front wall 626 of theater 600A. In fig. 6, viewing position 642 is below the midpoint height of LF display 620 within viewing volume 650, but in other configurations viewing position 642 may be above the midpoint. Depending on the configuration of the LF display system and/or the movie theatre, viewing volumes other than the one shown are also possible.

Fig. 6A shows the projection angle of holographic content presented from the display surface of LF display 620. Front wall 626, ceiling 628, floor 629 and rear wall 627 of theater 600A are shown, but the theater also contains two side walls. In the illustrated example, the rays are optimized for the geometry of the theater 600A. Here, the light rays (or rays, light, etc.) represent one or more of the projection paths of the holographic content projected from the display surface of LF display 620. The light rays may be projected from one or more of the energy surface locations on the display surface. For example, light rays representing a first portion of holographic content are projected from a first subset of energy surface locations on the display surface (e.g., the "top" of the LF display) along a plurality of projection paths. Similarly, light rays representing a second portion of the holographic content are projected from a second subset of energy locations on the display surface (e.g., the "bottom" of the LF display) along a plurality of projection paths.

In the configuration of fig. 6A, LF display 620 produces top ray group 602 and bottom ray group 608 from the top and bottom of LF display 620, respectively. Top ray group 602, centered on deflected ray 610, is projected from the top of LF display 620 at deflected angle 604. The deflection angle is the angle at which some holographic content can be projected from the display surface. The deflection angle may be different from the original projection angle of the holographic content. For example, as described above, the LF display system may generate holographic content for projection at an original projection angle (e.g., perpendicular to display surface 620), and the LF display system (i.e., as in theater 600 a) modifies the light rays of the holographic content (e.g., through optics, films, waveguides, etc.) to project the holographic content at deflection angle 624.

The deflection angle 604 can be defined in a variety of ways. For example, the deflection angle may be (i) an angle between the deflected ray 610 with respect to the LF array surface normal 606, (ii) an average angle of multiple projection paths of a portion of the holographic content projected from one or more energy locations, (iii) an intermediate angle of multiple projection paths of a portion of the holographic content projected from one or more energy locations, or (iv) another suitable definition of the deflection angle 604. Furthermore, as described above, the deflection angle 604 may be measured relative to the original projection angle of the holographic content. The original deflection angle can be defined in a number of ways. For example, the original projection angle may be (i) a measure (e.g., an average, a median, etc.) of a plurality of projection paths of a portion of the holographic content that is not projected at the projection angle, (ii) a normal to the display surface, (iii) a measure of a path projected from an energy device that generated the holographic content, or (iv) another suitable definition of the original projection angle.

In another example, the original projection angle and deflection angle 604 may be defined relative to the optical axis (or axes) of the LF display system. The optical axis may be defined in a number of ways. For example, the optical axis may be (i) a direction of a propagation path of a light ray projected from an energy surface location on the display surface, (ii) an average energy vector of all light rays projected from the energy surface location, (iii) a light ray (e.g., a centermost light ray) about which the projected holographic content is symmetric (e.g., rotationally, horizontally, vertically, etc.), or (iv) another suitable definition of an optical axis.

The optical axis may be the original optical axis or the deflected optical axis. The original optical axis is the optical axis of the holographic content projected at the original projection angle. The deflected optical axis is the optical axis of the holographic content projected in the deflected projection path. In this way, the deflection angle 604 is the angle between the original optical axis and the deflected optical axis.

Deflection angle 604 is oriented downward to a viewing position 642 in theater 600A. Bottom ray group 608, centered on orthogonal ray 612, is projected straight forward (i.e., without deflection angle) from the bottom of LF display 620 toward the rear of theater 600A. In other words, bottom ray group 608 is centered on a ray (shown as orthogonal ray 612) projected perpendicular to the surface of LF display 620. Orthogonal rays 612 from the bottom of the LF display 620 and deflected rays 610 from the top of the LF display 620 are shown in different directions, but in some examples, the two rays may be in substantially similar directions.

The angular spread of the top ray group 602 about the deflected rays 610 is the upper vertical field of view 614. The angular spread of the bottom ray group 608 about the orthogonal ray 612 is a lower vertical field of view 616. As shown, the angular spread of the upper and lower vertical fields of view 614, 616 is substantially the same, but may be different. The upper vertical field of view 614 and the lower vertical field of view 616 overlap to form a viewing volume 650 in the theater 600A. The horizontal field of view is formed by groups of rays projected from the left and right sides of the LF display 620. An example of a horizontal field of view is shown in fig. 4E.

In some embodiments, the light rays projected from a location between the top and bottom of LF display 620 may be defined by rays whose angles relative to surface normal 606 vary between deflection angle 604 at the top of the display and zero at the bottom of the display corresponding to the normal formed by orthogonal ray 612. In other words, rays projected from different points on LF display 620 may deflect at different angles between deflection angle 604 and normal 606 of LF display 620. In some configurations, the change in deflection angle may be a gradient across the surface of LF display 620. For example, a light ray projected from the middle of LF display 620 (e.g., ray 618) may be projected at an intermediate deflection angle 622. Intermediate deflection angle 622 has a value between zero degrees (since orthogonal ray 612 does not have a deflection angle) and deflection angle 604. An advantage of the gradient deflection angle is that for a fixed display FOV, the viewing volume of the holographic object projected from LF display 620 can be maximized.

Additionally, holographic content projected from display surface 620 at different deflection angles may be viewed from different fields of view. For example, a first portion of holographic content is projected at a first deflection angle from an energy surface location near the top of, for example, display surface 620. Similarly, a second portion of the holographic content is projected at a second deflection angle from an energy surface location near, for example, the middle of the display surface. In this case, depending on the first and second deflection angles, a first portion of the holographic content may be perceived from the first field of view and a second portion of the holographic content may be perceived from the second field of view. The first and second fields of view may be similar or different depending on the configuration of the LF display and the desired deflection angle.

In some configurations, the optical layer deflects the projected light rays at a deflection angle. In some examples, the optical layer may be located in front of a waveguide layer on the display surface. In this case, the deflection angle may be a function of the position of the optics on LF display 620. In various embodiments, the optical layer may comprise: a refractive optical layer comprising prisms having different characteristics; glass layers with different refractive indices or mirror layers, thin films, diffraction gratings, etc. The optical layers may be optimized for a particular viewing geometry and coupled to the LF display surface, allowing customization of the viewing volume at relatively low cost. In another configuration, the energy waveguide layer deflects the projection light rays exiting the waveguide at a deflection angle. The waveguide may be tilted, contain tilted sections or facets, or contain multiple refractive elements with tilted optical axes.

Fig. 6B shows a theater 600B that includes an LF display system in accordance with one or more embodiments. The array of LF display modules forms the polymeric surface of the LF display 620 on the front wall of the theater. Although not shown, some or all of the ceiling, some or all of the floor, or both may also be lined with the LF display 620. Theater 600B includes several viewing positions 642 arranged in rows and positioned to view holographic content presented by LF display 620. In fig. 6B, the LF display system is an embodiment of the LF display system 500. Further, the LF display modules in LF display 620 are embodiments of the LF display modules of LF display assembly 510.

The LF display system of theater 600B does not typically incorporate varying angles of ray deflection similar to theater 600A as depicted in fig. 6A. That is, in fig. 6B, there are no deflected rays (e.g., deflected ray 610) at a deflection angle (e.g., deflection angle 604). Top ray group 602 is centered on surface normal 606 and bottom ray group 608 is centered on orthogonal ray 612. In other words, the group of light rays from each location of the display surface are projected symmetrically about the normal to the display surface. The result is that the holographic viewing volume is positioned symmetrically to the center of the display and the LF display system is not optimized to present holographic content to all viewing positions 642 in the movie theater. For example, in the illustrated configuration, LF display 620 presents holographic content to a viewing volume 655 that does not contain any of viewing positions 642.

Thus, a theater that is capable of projecting light at a deflection angle increases the overall field of view of the LF display system (relative to a theater that is not capable of deflecting light). Similarly, when the rays are projected at a deflection angle, the separation between the viewing volume and the LF display decreases and the proximity between the holographic object volume and the viewing volume increases. The reasons for these improvements are similar to those described with respect to fig. 4C and 4D and the surrounding surface. In other words, projecting light rays at a deflection angle actually creates an enveloping surface from a completely planar surface.

In various examples, the LF display may be configured to deflect light in the infrared, visible, or ultraviolet electromagnetic spectrum. In some cases, LF display systems may be configured to employ deflection optics for deflecting light rays from one or more of those ranges. The LF display system may also deflect other types of energy. For example, if the polymeric surface is a dual energy surface that projects both electromagnetic energy and ultrasonic energy, the layer of acoustic material may be used to deflect sound at different angles. Deflected sound waves may achieve similar benefits to those described for deflected light rays. Some materials that may be used to deflect acoustic waves may include acoustic metamaterials that exhibit negative acoustic impedance, or other materials with appropriate acoustic characteristics.

Fig. 7A shows a perspective view of a theater 700 incorporating an LF display system in accordance with one or more embodiments. The front wall 710, side walls 712, and floor 714 of the theater 700 are shown, but the theater 700 also contains another side wall, ceiling, and rear wall. Here, the front and side walls (e.g., side wall 712) of theater 700 are lined with an array of LF display modules. The array of LF display modules forms a polymeric surface of LF display 720. Although not shown, some or all of the ceiling, some or all of the floor, or both may also be lined with the LF display 720. Viewing position 642 in the theater is positioned such that a viewer in viewing position 642 can perceive the movie and/or holographic content presented from LF display 720. In fig. 7A, the LF display system is an embodiment of the LF display system 500. Further, the LF display modules in array 720 are embodiments of LF display modules 512 within LF display assembly 510. The LF display system may include deflection optics as described in fig. 6.

Fig. 7B shows a perspective view of the theater 700 of fig. 7A with holographic content 722 rendered from the LF display 720. In this example, theater 700 is showing the movie "Jungle Book II: shere Khan's change ". Thus, holographic content 722 contains a jungle scene projected by LF display 720 in front of and to the side of the viewer at viewing position 642 in theater 700. Screening holographic video from multiple display surfaces surrounding the viewer results in a more immersive experience than displaying the video from a single surface in front of the viewer. In this case, the publisher of the movie has provided holographic content 722 rendered from LF display 720 while the movie theater 700 is showing the movie. That is, the jungle scenes are provided by the publisher of the movie, stored in a data store (e.g., data store 522), accessed by the LF display system, and then presented on the wall of the theater 700 by the LF display 720. In another example, a movie publisher sends holographic content over a commercial network (e.g., network 552) as vectorized data that is independent of the geometry of the receiving theater location. The LF display system 500 of the theater 700 decodes the vector quantized data and generates holographic content for the particular geometry and resolution of the theater's LF display 720.

Fig. 8 shows a perspective view of a movie theater 800 containing an LF display system presenting holographic content in accordance with one or more embodiments. The front wall 810, side walls 812 and floor 814 of the theater 800 are shown, but the theater also contains another side wall, ceiling and rear wall. The front wall 810 and side walls (e.g., side wall 812) of theater 800 are lined with an array of LF display modules. The array of LF display modules forms a polymeric surface of the LF display 820. Additionally, some or all of the ceilings (not shown), some or all of the floors 814, or both may also be lined with the LF display 820. Cinema 800 includes several viewing positions 642 arranged in rows and positioned such that viewers in viewing positions 642 can perceive holographic content presented from LF display 820. In fig. 8, the LF display system is an embodiment of an LF display system 500. Further, the LF display modules in array 820 are embodiments of LF display modules 512 of LF display assembly 510.

Fig. 8 shows a movie theater 800 with holographic content 822 rendered from an LF display 820. In this example, the classical movie "Invasion of the AA Aliens! "is being projected from LF display 820 onto the front wall so that a viewer in viewing position 642 may view the movie. LF display 820 also presents holographic content 822 in the holographic object above viewing position 642. Here, array 820 presents holographic content 822 comprising several flying discs horribly floating through a holographic object volume, not shown but extending in front of and behind each LF display surface. LF display system 820 displays a discouraging flying disc because, for example, a movie received from a publisher contains a holographic content track for the movie. The holographic content track contains holographic content for simultaneous presentation in a cinema 800 showing a movie with an LF display system. Thus, when projecting a movie from array 820, the LF display system accesses corresponding holographic content 822 from a data storage device (e.g., data storage device 522) and renders the holographic content 822 in holographic object volumes that extend in front of and behind each surface of LF display 820. Holographic content 822 presented by the LF display system in movie theater 800, a flying saucer, can enhance the viewer experience by increasing the perception of "tension" and "coming the end of the day".

Fig. 9A and 9B show a movie theater 900 with an LF display system presenting holographic content to different groups of viewing locations in accordance with one or more embodiments. Fig. 9A shows holographic content presented to a first set of viewing positions in front of a movie theater, and fig. 9B shows holographic content presented to a second set of viewing positions behind the movie theater. The front wall 910, side wall 912, and floor 914 of the theater 900 are shown, but the theater 900 also contains another side wall, ceiling, and rear wall. The front wall 910 and side walls (e.g., side wall 912) of the theater 900 are lined with an array of LF display modules. The array of LF display modules forms a polymeric surface of the LF display 820. Additionally, some or all of the ceiling (not shown), some or all of the floor, or both may also be lined with the LF display 920. Movie theater 900 includes several viewing positions 642 arranged in rows and positioned such that viewers in viewing positions 642 can perceive holographic content presented from LF display 920. In this example, the viewing positions are divided into a first group 942A, a second group 942B and a third group 942C for convenience of description. In fig. 9A and 9B, the LF display system is an embodiment of the LF display system 500. Further, the LF display modules in array 920 are embodiments of LF display module 512 of LF display assembly 510.

The theater 900 also contains several different viewing sub-volumes. Due to the complexity of its shape, the viewing subvolume is not shown. The view subvolume is a subvolume of the view volume of LF display 920. As shown, the viewing volume of the LF display 920 is approximately the size of the movie theater 900. Thus, the viewing sub-volume is some portion of the volume of the movie theater 900. For example, the viewing subvolume can correspond to a volume that encompasses the first group of viewing positions 942A, the second group of viewing positions 942B, the third group of viewing positions 942C, or some combination thereof.

In this example, LF display 920 is projecting the movie "Basic cuts" so that a viewer in viewing position 642 can view the movie. In addition, all viewers in the theater 900 view the projected movie from the LF display 920 from the front wall 910 of the theater 900 in a similar manner. That is, holographic content presented from the front wall 910 of the theater 900 can be seen by all viewers in the theater 900. However, the LF display system in the theater is configured such that the holographic content projected from the LF display on the sidewall 912 is different for the viewing subvolumes enclosing the groups 942A, 942B and 942C. That is, the LF display system may present different holographic content to different viewing subvolumes. To illustrate, for example, a viewer at a viewing position in the third group 942C behind the movie theater 900 may perceive holographic content that may not be perceivable by a viewer at a viewing position in the first group 942A in front of the movie theater 900. This principle applies to any of the viewing sub-volumes in the movie theater 900.

Fig. 9A and 9B show an LF display system of a movie theater 900 presenting different holographic content to viewers in different viewing subvolumes.

For example, in fig. 9A, the LF display 920 on the sidewall 912 presents the holographic content 922 to viewers in a first group 942A (e.g., a first particular viewing sub-volume) in front of the movie theater 900. Viewers at viewing positions in other groups (e.g., 942B and 942C) in the movie theater 900 may not see the same holographic content 922. In this example, the LF display 920 presents holographic content containing a piece of sad field in the scotland wilderness to viewers in the first group 942A closest to the viewing position of the front wall 910 of the movie theater 900.

In fig. 9B, the LF display 920 presents the holographic content 923 to a third group 942C of viewing locations (e.g., a second particular viewing sub-volume) behind the movie theater 900, such that viewers in the viewing locations in the group can perceive the holographic content 923 generated for the viewers in the third group 942C. Viewers at viewing positions in other groups (e.g., 942A and 942B) in movie theater 900 do not see the same holographic content (e.g., they may instead view holographic content 922). In this example, the LF display 920 presents holographic content 923, including a group of zombies that are away from a haunted house of a sad field located in the wasteland wilderness, to a third group 942C behind the movie theater 900.

Presenting different holographic content (e.g., holographic content 922 or 923) to viewers in different view sub-volumes allows the LF display system to enhance the viewing experience of viewers in different view sub-volumes in different ways. For example, the cinema "ambience" in each viewing subvolume may be different. To illustrate, referring to fig. 9A through 9B, holographic content 923 presented to users in a third group 942C behind movie theater 900 is more intense than holographic content 922 presented to viewers in a first group 942A in front of movie theater 900. In this case, the movie theater may have different movie ratings for viewing positions in different viewing sub-volumes. For example, Basic handles may be rated PG for viewers in the first group 942A and R for viewers in the third group 942C. In another example, the audio content of the holographic content may be for different viewing subvolumes (e.g., using audio beamforming). Thus, for example, a viewer in the first group 942A may hear audio holographic content without dirty words, while a viewer in the third group 942C hears audio holographic content with dirty words.

Notably, fig. 6 to 9 show a similarly structured cinema. That is, each theater has a similar configuration, including a particular orientation of walls and floor and a particular positioning of viewing positions within the theater. The arrangement includes a substantially orthogonal front wall, two side walls, a rear wall and a ceiling. The arrangement also includes a sloping floor that rises in a gradient from the front wall to the rear wall. The LF display of the movie theater is configured such that the viewer primarily views holographic content presented from the front wall and/or side walls. The configuration includes viewing positions arranged in three sections, with each section including three rows of twenty seats. Each row of seats is positioned substantially parallel to the front wall. The viewing position of each section corresponds to a viewing volume. The configuration of the theatre in figures 6 to 9 is given as an example but many other configurations are possible. For example, any of the configurations in fig. 4C to 4F may be applied to an LF display system in a movie theater.

Displaying content to viewers in a movie theater

Fig. 10 is a flow diagram of a method 1000 for displaying holographic content to a viewer in a movie theater (e.g., movie theater 700, movie theater 800, or movie theater 900) in the context of an LF movie network (e.g., LF movie network 550). The method 1000 may include additional or fewer steps, and the steps may be performed in a different order. Further, various steps or combinations of steps may be repeated any number of times during the performance of the method.

LF display system 500 transmits 1010 a request for holographic content (e.g., of a movie and/or enhanced movie) to network system 556 over network 552.

In one embodiment, LF movie generation system 554 creates holographic content and transmits the corresponding holographic content to network system 556. In other embodiments, network system 556 may access previously generated holographic content from a data storage device. Network system 556 transmits the holographic content to LF display system 500 so that it can be presented to viewers in a movie theater.

LF display system 500 receives 1020 holographic content from network system 556 via network 552.

The LF display system 500 determines 1030 the configuration of the theater and/or LF display system. For example, LF display system 500 may access a configuration file containing several parameters describing the layout and/or configuration of a theater, as well as the hardware configuration of LF display system 500 contained in the theater. The parameters may include, for example, LF display FOV, panel configuration, panel resolution, seated viewing sub-volumes, holographic object volume geometry, volume haptic projection parameters, layout of 2D panels to enhance the movie, other available sensory devices, any other details of the movie theater hardware, and the like. To illustrate, the configuration file may contain the location, spacing, and size of the viewing locations in the theater. Thus, the LF display system 500 may determine that the viewing positions in the first three rows are in the first viewing subvolume, the viewing positions in the middle three rows are in the second viewing subvolume, and the viewing positions in the last three rows are in the third viewing subvolume. In various other embodiments, the LF display system 500 may determine any number and configuration of viewing sub-volumes at any location within the theater. In some cases, the number and configuration of viewing sub-volumes may be based on the size of the movie theater. For example, a small cinema may have fewer viewing positions in each viewing sub-volume than a large cinema. In some examples, the viewing sub-volume may be the entire movie theater or some other portion of the movie theater. In other examples, the content may be adjusted for different viewer viewers depending on instructions from the content provider.

The LF display system 500 generates 1040 holographic content (and other sensory content) for presentation on the LF display system based on the hardware configuration of the LF display system within the theater and the particular layout and configuration of the theater. In various examples, the movie may be displayed on a theater screen or from an LF display system in the theater. Determining holographic content for display may include rendering the holographic content appropriately for the movie theater or viewing sub-volume. For example, the LF display system 500 may enhance a movie for a particular viewing sub-volume by displaying interactive holographic content (e.g., viewer-touchable content) to the viewing sub-volume. Other viewing subvolumes may not have interactive holographic content.

The LF display system 500 renders 1050 holographic content in holographic object volumes in a movie theater such that viewers at viewing positions in each viewing volume perceive appropriate show content. As an example, a viewer in a particular viewer sub-volume perceives interactive holographic content, while a viewer in another viewer sub-volume does not perceive interactive holographic content. In addition, the LF display system may present different audio content such that viewers in different viewer sub-volumes perceive different audio content.

The LF display system 500 may determine information about viewers in the sub-volumes at any time while the viewers are watching the movie or the holographic content. For example, the tracking system may monitor the facial responses of viewers in the viewing sub-volumes, and the viewer profiling module may access information about the characteristics of viewers in those viewing sub-volumes.

The LF display system may create (or modify) holographic content for presentation to the viewer based on the determined information. For example, the LF processing engine may create a fireworks show for display during the credits of a movie based on the determined information about the viewers.

Additional configuration information

The foregoing description of embodiments of the present disclosure has been presented for purposes of illustration; it is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. One skilled in the relevant art will appreciate that many modifications and variations are possible in light of the above disclosure. Notably, the foregoing description describes the use of a light field display assembly to present holographic content to a viewer in a movie theater. However, the principles, methods, and systems described herein may be more broadly applied to presenting holographic content to viewers in various other locations. For example, the content may be displayed in an outdoor theater, on a billboard or advertisement, in a wall of a building, and so forth.

Some portions of this description describe embodiments of the disclosure in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Moreover, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combination thereof.

Any of the steps, operations, or processes described herein may be performed or carried out using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, the software modules are implemented in a computer program product comprising a computer readable medium containing computer program code, the computer readable medium being executable by a computer processor to perform any or all of the described steps, operations, or processes.

Embodiments of the present disclosure may also relate to apparatuses for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general purpose computing device selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a non-transitory tangible computer readable storage medium or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. Moreover, any computing system referred to in the specification may contain a single processor, or may be an architecture that employs a multi-processor design for increased computing capability.

Embodiments of the present disclosure may also relate to products produced by the computing processes described herein. This product may include information resulting from a computing process, where the information is stored on a non-transitory tangible computer readable storage medium and may include any embodiment of the computer program product or other data combination described herein.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the disclosure be limited not by this detailed description, but rather by any claims based on the application to which they pertain. Accordingly, the disclosure of the embodiments is intended to be illustrative, but not limiting, of the scope of the disclosure, which is set forth in the following claims.